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Overview 


The federal Office of Child Support Enforcement (OCSE) created the Behavioral Interventions for Child 
Support Services (BICS) demonstration in 2014. The goals of the demonstration were to use insights 
from behavioral science to develop interventions that could improve child support services and increase 
parent engagement, and to encourage the rapid-cycle testing of these strategies, leading to further 
improvements. In the process, the project aimed to build the participating child support programs’ 
capabilities in behavioral science and evaluation and support these and other child support programs 

in replicating and implementing effective practices. Child support agencies in California, Colorado, the 
District of Columbia, Georgia, Ohio, Texas, Vermont, and Washington participated as grantees. 


The project addressed several common operational challenges that may create barriers to the success- 
ful establishment, enforcement, and modification of child support orders. The child support process 
can be difficult for parents to navigate, for example, given the complex legal language and lengthy forms 
involved, and parents may perceive their interactions with child support agencies to be adversarial. The 
22 interventions, tested by the grantees in two phases, used a range of design principles from behavioral 
science — for example, simplification, personalization, and reminders — to address these challenges. 
One program simplified the initial paperwork sent to the parents required to pay child support, pro- 
viding clearer language and instructions on next steps. Another program held in-person meetings with 
parents to help them understand their payment options and make plans for ongoing payment. A third 
program assigned particular staff members to help parents navigate the process of requesting modifica- 
tions of their child support order amounts and complete the necessary forms. 


Findings from the demonstration include the following: 


= Outreach and communication informed by behavioral science can make parents more likely to re- 
spond to outreach, attend in-person meetings, submit required forms, and make initial payments on 
new orders. 


= Interventions directed at a specific activity, such as encouraging parents to attend a meeting at the 
child support office, can affect outcomes related to that activity, but will not necessarily affect other 
outcomes at later points in the child support process, such as longer-term payment rates. 


= The interventions’ effects are typically modest in size, but they are also inexpensive to implement. 


= Interventions informed by behavioral science do not always have the intended effects, highlighting 
the need to continue testing and refining practices. 


The BICS project demonstrates the significant potential for behavioral science to improve child support 
services for parents and staff members. Each of the participating agencies has since incorporated con- 
cepts from behavioral science into ongoing program operations. In addition, the grantees, along with 
OCSE, are mentoring additional states and localities as they attempt to improve their services and meet 
their goal of securing support for children. 
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INTRODUCTION 


he purpose of the child support program is to secure financial support for chil- 
dren whose parents live apart. The program served over 15 million children and 
their families in 2017. The federal Office of Child Support Enforcement (OCSE) in 
the U.S. Department of Health and Human Services partners with state, tribal, 
and local child support agencies to promote effective child support services. Core child 
support program functions include establishing paternity, establishing and enforcing child 


support orders, and collecting and distributing payments. 


Under federal oversight, states, territories, and tribes administer the child support program in their 
jurisdictions, and there is substantial variation among jurisdictions (called states throughout, for 
simplicity) in how they do so. Guided by federal and state laws and regulations, as well as judicial 
and administrative structures, child support processes are typically complicated and legalistic. 


These processes can be confusing and opaque for parents, which may discourage parents from 
participating in them. The materials, forms, and communications that parents receive often in- 
clude legal terms that are difficult to understand. Overwhelmed by the legal terminology and 
volume of information, parents may not comprehend the steps they need to take to fulfill child 
support requirements. Parents may also have negative perceptions of the child support program, 
particularly if their initial interactions with it cast them as defendants in court cases. In this context, 
parents’ decisions and actions may not be in their best interest or their children’s, and the conse- 
quences of these actions can have long-lasting legal, financial, and emotional impacts on families. 


All people tend to become overwhelmed when faced with large amounts of information. We 

have limited attention spans, procrastinate, avoid activities we view as punitive or that engender 
negative feelings, and forget to follow through on obligations.’ Behavioral science sheds light on 
these basic human tendencies and can help explain why people act the way they do within a given 
system or program. This information can help program designers identify which program features 
create behavioral “bottlenecks,” or points in the process with notable declines in participation, and 
develop solutions. 


OCSE created the Behavioral Interventions for Child Support Services (BICS) demonstration in 
2014, awarding grants to child support agencies in California, Colorado, the District of Columbia, 
Georgia, Ohio, Texas, Vermont, and Washington. OCSE had several goals for BICS: 


= Use insights from behavioral science to develop interventions that may improve child support 
services and increase parent engagement 


= Encourage rapid-cycle tests that may lead to further improvements in child support services 
= Build grantees’ capabilities in behavioral science and evaluation methods 


1 See, for example, Thaler and Sunstein (2008). 
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= Support state child support agencies in expanding or replicating successful interventions 
= Provide the broader child support community with findings and lessons learned 


In addition to the demonstration grants, OCSE also awarded an evaluation grant to the State of 
Washington’s Division of Child Support (DCS). DCS issued a contract to MDRC (in collaboration with 
MEF Associates and the Center for Policy Research) to form a Technical Assistance and Evaluation 
(TAE) team that provided overarching support to grantees in each of three phases of the demon- 
stration (described below). 


In the first of the BICS demonstration’s three phases, each grantee worked closely with the TAE 
team to identify a problem of interest, conduct a process to define goals called “behavioral di- 
agnosis and design,” implement at least one intervention, and evaluate it, leading to 12 separate 
evaluations.” The Phase | interventions were evaluated using random assignment research designs 
that examined effects on short-term measures of engagement, such as attendance at meetings or 
payments over the few months after the intervention. The grantees had a more independent role 
in Phase Il, using the early findings from Phase | to develop new interventions. Six of the ten inter- 
ventions in Phase II were evaluated using random assignment, three were evaluated using other 
methods such as nonexperimental comparison groups and implementation studies, and one was 
not evaluated at all. In the third phase, called the Integration Phase, grantees worked closely with 
OCSE and the Washington Division of Child Support to develop additional interventions as well as 
plans for integrating elements of behavioral science into their standard practices moving forward. 


This report describes the BICS interventions and their findings. It presents major operational 
challenges in child support and several tools informed by behavioral science that were used by 
the participating child support agencies to address those challenges. The report then provides an 
overview of findings from each of the evaluated interventions and lessons for the child support 
community and the broader field of behavioral science. 


OVERVIEW OF CHILD SUPPORT AND COMMON 
OPERATIONAL CHALLENGES 


Child Support Overview 


Child support involves several core functions: (1) opening cases and, if necessary, establishing 
paternity; (2) establishing child support order amounts; (3) collecting and processing payments 
and enforcing child support obligations; and (4) modifying orders over time, if necessary. In some 
states, child support is managed primarily through judicial processes (involving courts and court 
officials) while in others it is managed primarily through administrative processes (involving state 
or county agencies and staff members). Some states have a combination of both. Although the spe- 
cific processes vary across states, they typically have certain steps in common. The descriptions 
below are illustrations of these steps and are meant to reflect the child support program broadly. 
They may not be accurate for every state. 


2 Some grantees tested more than one intervention in the first round of tests. 
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= Opening cases and establishing paternity. Most cases open when a parent applies for sup- 


port directly or applies for a public benefit that triggers an automatic referral to the child sup- 
port agency (for example, cash assistance through the Temporary Assistance for Needy Families 
program, and in some states, Medicaid, food assistance through the Supplemental Nutritional 
Assistance Program, and subsidized child care). In many states, staff members attempt to locate 
and make contact with the custodial and noncustodial parents.? Depending on the state, this 
contact may consist of a phone call, a letter, an invitation to an in-person meeting, or a com- 
bination of these methods. Staff members also determine whether paternity has been estab- 
lished and work to establish paternity if needed. 


Notifying parents and establishing orders. Child support agencies make contact with 
custodial and noncustodial parents to begin the order establishment process, part of which 
involves determining the appropriate monthly order amount.‘ States take various approaches 
to this step, although parents are typically provided notice through a formal legal mechanism 
known as “service of process,” commonly called “service.” The service package, often deliv- 
ered by certified mail — or, in some cases, by a law enforcement officer, private contractor, or 
in-house process server — notifies the noncustodial parent that he or she has been named as 
the parent of the child and that a legal proceeding has been initiated. This notice typically gives 
the noncustodial parent information about the pending legal action and requirements for a 
response to that legal action. In some states, the child support agency may invite parents to a 
meeting in order to try to reach agreement on the order amount. In some states the notice may 
include a proposed child support order amount, and if neither parent contests that amount, it 
goes into effect by default. 


Collecting and processing payments and enforcing obligations. Once an order is estab- 
lished, the noncustodial parent is responsible for payments. If the child support agency is aware 
of the parent’s employer, the agency sends an income-withholding notice to the employer and 
the employer will begin withholding payments from the parent’s earnings and submit them to 
the child support agency. If payments are not being received through income withholding, the 
noncustodial parent must make payments through the child support agency. The payments 
are then disbursed to the custodial parent.5 Child support agencies have a variety of options to 
enforce child support orders when noncustodial parents do not pay what they owe. They can 
put liens on property, in which the property can be taken if the child support debt is not paid; 
suspend drivers’ licenses; seize personal funds in bank accounts; intercept tax refunds; or initi- 
ate civil contempt proceedings that require parents to appear in court to determine next steps. 


Modifying orders, if appropriate. When there is a substantial change in a parent’s income or 
living situation, either parent can request that the order be reviewed for possible modification. 
While all states have formal processes for case review and modification, those processes vary. 


3 


Noncustodial parents are also sometimes called obligors; they are the parties who have been ordered to pay child 
support. Parents who receive child support are described as “custodial parents.” 

All states have guidelines that are used to set order amounts based on both parents’ income and other factors. For 
more information, see National Conference of State Legislatures (2019). 

When the custodial parent is receiving cash assistance, some or all of child support paid may go to the state as reim- 
bursement for those public-assistance costs. 
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Usually, the modification process aligns the order with the state’s support guidelines, and it is 
the responsibility of the parent seeking modification to contact the agency to adjust the order. 


Common Operational Challenges 


Agencies face many challenges as they establish, enforce, and modify child support orders. As BICS 
grantees identified problems and developed interventions, they focused on operational issues and 
challenges that appeared to be common to multiple child support agencies. 


= One or both parents can perceive the initial contact as adversarial. Custodial parents 
may have negative views of the child support program if they receive public benefits and are 
thus mandated to cooperate with the program or face penalties. Noncustodial parents may 
also be likely to feel like child support is against them as they are commonly in the position of 
responding to legal and administrative actions imposed on them by the child support program. 
One of the first communications some parents receive from a child support agency comes as 
part of a service package and is framed as a lawsuit against them, casting them as litigants rath- 
er than parents and emphasizing the sanctions they will face if they fail to comply. In Georgia, for 
example, the summons may be delivered to the parent at home or at work by a law enforcement 
officer. In California, the first page of the service package names the parent as a defendant ina 
court case. 


= Lengthy forms and complex legal language may create confusion and avoidance. Initial 
mailings or meetings with staff members often present parents with overwhelming amounts of 
information in terms that are difficult to understand. The Vermont service package is dense and 
uses legal terms such as “plaintiff,” “defendant,” and “parentage,” to describe the parents and 
their statuses. The forms in Ohio are similarly dense with legal terms and use the phrases such 
as “health insurance obligor” and “child support obligor” to refer to the noncustodial parent. 
Research has shown that individuals are more likely to understand and comply with information 
when it is provided in plain language rather than in legal language.® The length of the commu- 
nications can also be a hurdle — the service package in California, for example, is 41 pages, and 
the package in Washington is roughly 15 pages. 


= Written communication can bury useful and relevant information. Child support agen- 
cies may be legally required to convey a lot of information to parents, sometimes in a limited 
amount of space and time, and the information that parents need the most is often buried or left 
out entirely. In California, for example, the form that parents must review and complete if they 
want to request a hearing or contest an order amount does not appear until page 26 of 41 in the 
service package. During order establishment meetings in Texas, child support staff members 
make a priority of discussing the steps for setting order amounts and completing legal docu- 
ments. As a result, parents may leave those meetings knowing how much they are now required 
to pay but not how to make payments. 


= Child support processes can be difficult to navigate. The processes for establishing orders, 
modifying them, and setting up and making initial payments can be burdensome. In Vermont, 


6 Kimble (2012). 


for example, the service package is sent to noncustodial parents by certified mail, which means 
they must go the post office if they are not home to receive it. In Colorado, there are at least 
eight different ways to make payments, some of which charge fees, but the staff has no set 
process to help parents understand or navigate these choices. In Ohio, obtaining a modification 
requires two rounds of paperwork from parents and can take more than three months. For the 
review, parents must provide W-2s and tax forms for the last three years and pay stubs for the 
last six pay periods, in addition to filling out an eight-page application documenting their em- 
ployment, income, deductions, and health insurance. 


BEHAVIORAL SCIENCE AND THE BICS INTERVENTIONS 


Policymakers and program designers often operate in a framework based on the assumption that 
individuals will have the time and capacity to review and understand relevant information and 
make informed, rational decisions, comparing the benefits and costs of their actions. If that were 
the case, however, the operational challenges outlined above would not present such large barri- 
ers to parents’ participation in the child support process. Behavioral science has highlighted that 
people do not always behave as fully informed, rational actors for a variety of reasons. 


Based on work in economics, psychology, and other social sciences, behavioral science offers a 
more realistic view of how individuals make decisions and illustrates that even small program chal- 
lenges can create large hurdles to participation for the individuals the programs are designed to 
serve. In fact, a substantial body of research demonstrates that interventions informed by behavio- 
ral science can have effects in a variety of contexts, such as retirement savings, financial aid appli- 
cations, medication adherence, child care subsidy application renewals, and healthy eating.’ 


In the context of child support, interaction with the program can involve multiple, lengthy steps. 
At each step, people may put off action and sometimes eventually forget to act. Parents may 

also exhibit avoidance — ignoring behaviors or information deemed unpleasant, such as punitive 
communications from child support or negative interactions with the other parent. All people, but 
especially those under financial stress, have a limited ability to navigate and engage with compli- 
cated systems, so lengthy and legalistic communications from child support may be challenging 
for parents to understand.® 


These behavioral tendencies can have negative effects on the individuals and families involved, 
and on the programs that serve them, so that programs cannot carry out in practice what their 
policies intend. Behavioral science offers several principles that can be used to improve program 
design and operational efficiency. These concepts of behavioral science were used in BICS: 


= Simplification: Shorten and streamline materials, communications, and processes when pos- 
sible, given individuals’ limited capacity to absorb and process complex information. 


7 Forexample, see Bettinger, Long, Oreopoulos, and Sanbonmatsu (2012); Schwartz (2007); Madrian and Shea (2001); 
Volpp et al. (2008); Bhargava and Manoli (2015); Richburg-Hayes, Anzelone, and Dechausay (2017). 
8 Mullainathan and Shafir (2013). 
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= Personalization: Nudge people toward particular behaviors through in-person assistance or 
personalized written communication. 


= Implementation prompts: Connect individuals’ intentions with their actions by helping them 
identify and plan next steps. 


= Loss aversion: Emphasize the losses associated with inaction, given that individuals tend to 
prefer avoiding losses over acquiring gains of a similar size.® 


= Identity priming: Encourage individuals to focus on one of the many roles that define them 
(for example, “parent”), to motivate them to engage in behaviors associated with that role. 


= Reminders/follow-up: Provide reminders through repeated contact, to help people overcome 
procrastination and barriers to taking next steps. 


= Dates and deadlines: Make dates prominent and set deadlines to call attention to action and 
mitigate the tendency to procrastinate. 


= Social influence: Promote actions by conveying how society, peers, or a person of influence 
would act. 


The BICS Interventions and Findings 


Each of the BICS grantees developed interventions using one or more of these design principles. In 
most cases, materials and processes were simplified. In some interventions, the losses associated 

with inaction were also highlighted, and in others, individuals were primed to think of themselves 

as parents. Several interventions included reminders to take next steps or make payments. 


In addition to behavioral science, a few interventions also used principles from the field of proce- 
dural justice, which is based on the idea that individuals’ perception of the fairness of an adminis- 
trative or legal process and how they are treated during the process affects how they respond to it. 
The central principles of procedural justice are respect, neutrality, helpfulness, individual under- 
standing, and individual voice. This approach has produced significant increases in compliance and 
long-term rule-following behavior in criminal justice and judicial settings and could also be benefi- 
cial in the child support context.” 


Figure 1 presents a general summary of the grantees and their interventions. The grantees are 
located across the country, and the interventions occur at different points in the child support pro- 
cess and use different methods. 


Most interventions were evaluated using random assignment designs, in which eligible cases or 
parents were assigned at random to the intervention group, who received the intervention, or 
the control group, who received business-as-usual services (see Figure 2). Random assignment 


9 Tversky and Kahneman (1991). 
10 Berman and Gold LaGratta (2012). 
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Figure 1. Map of BICS Locations 
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Figure 2. Random Assignment Diagram 
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provides the most credible evidence of intervention effects. Each section of this report includes 
an impact table presenting estimated effects on targeted outcomes and other outcomes (hypoth- 
esized in advance to be affected by the intervention) for those interventions that were evaluated 
using random assignment. Only effects that are statistically significant, marked with asterisks, are 
considered effects of the intervention." With the exception of the interventions in Washington, DC, 
which were not previously described in a stand-alone, published brief, findings from interventions 
that were evaluated using other methods are discussed briefly in the text but are not presented in 
the impact tables. These findings are considered suggestive only. 


Each of the interventions was guided by a theory of change, which hypothesized that changes 
informed by behavioral science affecting a specific part of the child support process would have 
effects on one or two near-term, targeted outcomes. Examples of targeted outcomes include the 
percentage of parents who responded to outreach or who attended meetings, and the percentage 
of cases where parents reached agreements on orders. Most agencies in the demonstration also 
believed that positive effects on targeted outcomes would lead to positive effects later in the child 
support process and ultimately on longer-term payment rates. However, since BICS emphasized 
short evaluation time frames, its findings provide limited evidence on longer-term outcomes. Only 
a few tests estimated interventions’ effects on outcomes beyond the near term or on payment 
rates beyond the first few months. 


The project team published briefs describing each of the Phase | interventions and their findings. 
One-page summaries of all interventions (in Phase | and Phase II) are presented in Appendix A. The 
discussion below summarizes the interventions and their findings. It is organized in sections titled 
according to the step in the child support process in which the intervention took place. Each inter- 
vention is identified by the state abbreviation of the grantee and the order in which the grantee im- 
plemented the intervention. (For example, the first Vermont intervention implemented is identified 
as VT1.) Each section presents a general summary of the issue each intervention hoped to address, 
its central components, and its effects, if any. 


Opening Cases: Custodial Parent Responding to Outreach/Attending Intake Meetings 


The three interventions in this group, all located in the District of Columbia (DC), are the only ones 
that specifically targeted custodial parents (see Tables 1 and 2). They addressed two issues. First, 
some custodial parents who are eligible for child support services may not know that they are eligi- 
ble and may not engage with the agency. Two of the interventions (DC1 and DC2) targeted custodial 
parents receiving Medicaid. These parents, whose cases had been referred to child support years 
ago but whom the child support agency had never reached, were eligible for services but had never 
applied for help to pursue paternity or child support. Because these parents’ cases were relatively 
old, the child support agency hoped to engage with them either to close the cases or to pursue 
services on their behalf. 


The second issue, targeted by the DC3 intervention, is that many recipients of Temporary As- 
sistance for Needy Families (TANF) are eligible for and required to cooperate with child support 
services but do not take all the steps necessary to start the process. In particular, despite receiving 


11 Differences in outcomes between the program and control group that are statistically significant are unlikely to have 
arisen by chance. 
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Table 1. Summary of BICS Tests Related to Opening Cases 


Responding to Outreach/Attending Intake Meetings 


Positive 
Issue Goal Test Intervention Effect 
Many low-income sin- _ Increase custodial DCI * Outreach letter informed x 
gle parents receiving parent contact with by behavioral science 
Medicaid are eligible the child support * Robocall reminder 
to receive child sup- agency, so the agency « Postcard sent to 
port services but do can either close par- nonresponders 
not engage with the ents’ cases or provide 
child support agency. services DC2 * Same as DC1, plus reminder Y 
texts with the option to re- 
spond by text 
Many custodial par- Increase the number DC3 * Appointment letter informed NA 
ents receiving TANF of custodial parents by behavioral science 
do not attend the who attend the intake ° Initial and reminder texts 
child support intake meeting with relevant e Reminders once per month 
meeting or supplythe documents for three months 


documents necessary 
to start the process. 


NOTE: The final column indicates whether the evaluation of that intervention detected positive and statistically significant 
effects on at least one outcome of interest. “NA” indicates that the intervention was not evaluated using a randomized 
controlled trial. The Washington, DC interventions are discussed in greater detail than other interventions that did not 
involve randomized controlled trials because they have not previously been described in a stand-alone brief. 


warnings of penalties if they do not comply, many parents do not attend required intake meetings 
or provide the paperwork needed to make their cases ready for court. 


All three interventions included outreach informed by behavioral science that provided parents 
with information on their options and encouraged them to take the next steps. The outcomes 
measured for the first two tests (DC1 and DC2) included the percentage of parents who responded 
to outreach, the percentage who submitted applications, and the percentage of cases where pater- 
nity was established. The second test (DC2), which provided the option for parents to respond by 
text message, was not evaluated using a randomized controlled trial, so only the findings from DC1 
are shown in Table 2. However, the results from DC2 suggest that the intervention that did include 
text messages may have had larger effects on parents’ responses. The third intervention (DC3), 
targeting parents receiving TANF, was not evaluated. 


Table 2. Impacts on Selected Outcomes Related to Opening Cases 


Responding to Outreach/Attending Intake Meetings 


Intervention Impact 
Outcome DC1 


Targeted outcome (%) 


Custodial parents respond to outreach 3.8** 


Other outcomes (%) 


Child support services application submitted 1.9** 
Paternity established -1.3 
Net cost (S) NA 


SOURCE: MDRC calculations are based on data from the Washington, DC Support Services 
Division. 
NOTES: Estimates in this table are regression-adjusted, controlling for pre-random as- 
signment characteristics. 

Statistical significance levels are indicated as: *** =1 percent; ** =5 percent; * =10 percent. 

Net costs were estimated for each of the interventions in Phase | tests except for 
Washington, DC and Ohio. 

“NA” indicates that costs were not estimated. 
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Highlighted Findings 


= Additional outreach informed by behavioral science can increase responses from cus- 
todial parents. 


The intervention that targeted Medicaid recipients (DC1) more than doubled the rate at which 
custodial parents made contact with the agency: 6.9 percent of parents in the intervention group 
made any form of contact with the agency compared with 3.2 percent of parents in the control 
group, for an impact of 3.8 percentage points. The second test (DC2) was assessed by comparing 
the outcome measures of a single group at two points in time. Offering the option to request child 
support services by text message increased contact from the first point in time to the second by 
10.7 percentage points (not shown in the table) more than tripling the existing rate. The latter inter- 
vention was not evaluated using random assignment, so the findings are only suggestive. 


= There were minimal effects on other outcomes beyond initial contact. 


There were small increases in applications submitted but no effects on the number of child sup- 
port requests submitted to court (not shown in the table) or on paternity-establishment rates. 
There may not have been effects on these other outcomes because no intervention elements were 
designed to affect these outcomes specifically. Alternatively, because these Medicaid-only cases 
were very old, having been referred to child support years ago, the parents may have been very 
unlikely to pursue support even with additional help. 
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= Itis challenging to reach custodial parents whose child support cases are very old. 


An important implementation challenge for the interventions that targeted Medicaid recipients 
with old cases was making phone contact with parents. Many of the phone numbers on file for 
parents were missing, disconnected, or incorrect. Robocall reminders did not go through for many 
parents in the DC1 test, and only one in five parents received text messages in DC2. 


Establishing Orders: Responding to Service 


The interventions in this category also focused on early engagement, but targeted noncustodial 
parents after child support cases were opened (see Tables 3 and 4). Specifically, they focused on 
the point at which parents are served notice that a child support order is being established. The 
goal was to increase the rate at which parents responded to this notice, increasing their engage- 


ment in the process. 


Table 3. Summary of BICS Tests Related to Establishing Orders 


Responding to Service 


Positive 
Issue Goal Test Effect 
Many noncustodial Increase the num- CAI ° Revised service package ‘i 
parents do not re- ber of parents who with explanatory cover sheet 
spond to the service respond to the listing next steps 
package, resulting in service package by » Answer Form moved forward 
orders being set by submitting Answer in the service package 
default. Forms and reduce the e Reminder phone calls 
number of orders set e Specialized staff members 
by default trained to deliver services to 
the intervention group 
CA2/ ° The same service-package N 
SJ? modifications as CAI, but 
no reminder phone calls or 
dedicated staff members 
Increase parent WAI * Telephone outreach before N 


engagement in order 
establishment 


service to provide an over- 
view of the process, guided 
by achecklist 

Follow-up letter if no phone 
contact 

Service package cover sheet 
with clear instructions 
Specialized staff members 
trained to deliver services to 
this group 


NOTES: The final column indicates whether the evaluation of that intervention detected positive and statistically signif- 
icant effects on at least one outcome of interest. 
*This test took place only in San Joaquin County. 
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Table 4. Impacts on Selected Outcomes Related to Establishing Orders 


Responding to Service 


Intervention Impact 


Outcome CA1 CA2/SJ? WA1 


Targeted outcomes (%) 


Answer Form submitted al a -1.3 

Order established by stipulation/agreement 3.2** 0.6 -1.8 
Order established by default 21. 4.0** 1.3 
Other outcomes 

Noncustodial parent served (%) Lia 4.3** -0.5 
Order established (%) 5.377" 3.4* 0.1 
Paid child support, Months 1-6 (%) 2.4** 2.4 1.0 
Total payments, Months 1-6 ($) 29* 30 -78 
Paid child support, Months 1-12 (%) 17 0.6 
Total payments, Months 1-12 ($) 92** -229 
Net cost ($) 11.06° NA 11.09 


SOURCES: MDRC calculations are based on data from the California State Department of Child Support Services 
Data Repository and the Washington State Division of Child Support. 


NOTES: Estimates in this table are regression-adjusted, controlling for pre-random assignment 
characteristics. 

Statistical significance levels are indicated as: *** =1 percent; ** = 5 percent; * = 10 percent. 

Cells that are filled gray indicate that impacts were not examined for that outcome. 

Net costs were estimated for each of the interventions in Phase | tests except for Washington, DC and Ohio. 
Vermont net costs were not available due to incomplete data. (Net costs are calculated as the total cost of the 
intervention per intervention group member minus the total cost of business as usual per control group member.) 

“NA” indicates that costs were not estimated. 

This Phase II test took place only in San Joaquin County. 

>This figure is the average of the per-participant costs in Sacramento ($7.65) and San Joaquin ($14.46) counties. 
For more details on county-specific costs, see Gaffney, Fishman, and Smith (2019). 


The interventions (CA1, CA2/SJ, and WA1) used outreach informed by behavioral science, simplified 
service package materials, and in one case made pre-service phone calls to parents. CA2/SJ tested 
a similar intervention as CAI, without the dedicated staff and follow-up phone calls. The states hy- 

pothesized that if noncustodial parents were more engaged, fewer orders would be set by default, 

more orders would be reached by agreement and would reflect parents’ ability to pay, and parents 
would be more likely to pay support. 


All three interventions assessed effects on early engagement. In the two tests in California, the 
main outcome was the percentage of parents who responded to service by submitting an Answer 
Form, which parents must submit if they would like to request a hearing to contest a proposed 
order amount. In Washington, the main outcomes were the percentage of parents who requested 
hearings late in the process (after their orders were already established), the percentage of orders 
reached by agreement, and the percentage of parents who made payments. 


Highlighted Findings 


= Simplified communications materials may increase parent engagement, particularly 
among parents in groups with challenges to engagement. 


Both California interventions provided explainer sheets and moved the Answer Form to the front 
of the service package. Both led to positive effects on order establishment and, in one location, 

an increase in payment rates. However, these effects may be due to unexpected increases in the 
percentage of parents who received service packages in the first place, probably because staff 
members serving the intervention group made greater efforts to reach them. These unexpected 
“service effects” make it difficult to say whether the components of the intervention informed by 
behavioral science themselves led to increased payments. The CA1 intervention also translated the 
explainer sheet for parents whose primary language was Spanish, and it had very large effects for 
those parents. This finding suggests that increasing access through translation is an easy way to 
increase engagement. 


= Additional outreach and information have limitations. 


The intervention in Washington consisted of pre-service outreach and a cover sheet added to the 
service package. The evaluation did not find that this intervention affected engagement. There are 
at least two possible reasons. First, the pre-service phone calls may have been too far in advance 
of the process of setting orders and making payments to affect those outcomes. Parents may have 
needed additional follow-up calls as reminders or more assistance taking next steps. Second, the 
intervention may not have reached enough parents, particularly those who were less likely to be 
engaged. As noted in the next bullet, only about half of the parents had contact with the child sup- 
port agency before being served. 


= Limited contact information can make it challenging for staff members to reach par- 
ents before service. 


In the Washington intervention, the child support agency only had valid phone numbers for about 
a third of parents when the intervention started, so the staff had to conduct additional research to 
obtain contact information. Staff members eventually reached 36 percent of the parents by phone 
before service, although another 14 percent of parents called the agency after receiving a voice mail 
message or letter. 


Establishing Orders: Attending In-Person Meetings 

The interventions in this category had the goal of encouraging noncustodial parents to come into 
the child support office for an in-person meeting, either to accept service voluntarily or to partic- 
ipate in an order-setting meeting (see Tables 5 and 6). These states hoped that engaging parents 
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Table 5. Summary of BICS Tests Related to Establishing Orders 


Attending In Person Meetings 


Issue 


Many parents do not 
attend the initial 
order establishment 
meeting and do not 
reach agreement on 
orders with the other 
parent? 


Few parents accept 
service voluntarily 
in the child support 
office 


Goal Test 


Increase the number VTl 
of cases in which both 
parents attend an 

initial order estab- 

lishment meeting (or 
modification meet- 

ing in Vermont) and 

reach agreement on 

an order 


VT2 


TX2 


Increase the number GAI 
of parents who accept 
service voluntarily by 
coming into the child 
support office 


Intervention 


Invitation to the resolution 
meeting informed by behav- 
ioral science 

Reminder calls before the 
meeting 

Meeting held using proce- 
dural justice principles to 
help parents reach agree- 
ment on the order 


Similar intervention to VT, 
plus $25 transportation cards 
to meeting participants for 
gas or public transit 


Outreach to attend the meet- 
ing informed by behavioral 
science 

Mail and email reminders 


Redesigned initial notice 
encouraging parents to 
attend an in-person meeting 
to accept service, sent in an 
eye-catching envelope and 
including a calendar magnet 
Reminder flyer one week 
later 

In-person meeting using 
principles of behavioral sci- 
ence and procedural justice 


Positive 
Effect 


Y 


NA 


NA 


NOTES: The final column indicates whether the evaluation of that intervention detected positive and statistically 
significant effects on at least one outcome of interest. “NA” indicates that the intervention was not evaluated using a 
randomized controlled trial. 

*In Vermont, this initial meeting is referred to as a Case Manager Conference. 


early would start the parents’ interaction with child support on a more positive note, reduce the 
number of orders set by default, and ultimately increase payment rates. 


In Vermont, for example, both parents are expected to attend a case management conference at 
the court, but most do not. The VT1 intervention used mailings informed by behavioral science to 
encourage parents to come into a local office for a newly designed resolution meeting that took the 
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Table 6. Impacts on Selected Outcomes Related to Establishing Orders 


Attending In Person Meetings 


Intervention Impact 


Outcome GA1 VT1 


Targeted outcomes (%) 


Both parents participated in resolution meeting 8.4*** 
Order established by stipulation/agreement 13 
Accepted service voluntarily in person 8.2*** 

Other outcomes 

Order established (%) 1.5 

Order set to amount lower than default amount (%) 0.6 

Total payments, Months 1-3 ($) 15 


Cost calculations 


a 
Net cost (S) 9.02 unavailable 


SOURCES: MDRC calculations are based on data from the Georgia Division of Child Support Services and 
the Vermont Office of Child Support internal management information system. 


NOTES: Estimates in this table are regression-adjusted, controlling for pre-random assignment charac- 
teristics. 

Statistical significance levels are indicated as: *** =1 percent; ** = 5 percent; * = 10 percent. 

Cells that are filled gray indicate that impacts were not examined for that outcome. 

Net costs were estimated for each of the interventions in Phase | tests except for Washington, DC and 
Ohio. Vermont cost calculations were not available due to incomplete data. (Net costs are calculated 
as the total cost of the intervention per intervention group member minus the total cost of business as 
usual per control group member.) 

"This cost estimate is different from the Georgia cost reported in Anzelone, Timm, and Kusayeva (2018) 
because this estimate includes staff time as well as the cost of materials. The cost estimate presented 
in Anzelone, Timm, and Kusayeva (2018) presents the total cost of the materials per person for the in- 
tervention group. 


KKK 


place of the court meeting. Staff members who conducted the resolution meetings were trained to 
use principles from behavioral science and procedural justice to help parents reach a stipulation 

— agreement by both parents. The expectation was that if both parents attended a meeting, they 
would be more likely to reach an agreement that represented each of their interests and avoid a 
time-consuming and costly court hearing. 


The GA1 intervention focused on noncustodial parents and encouraged them to accept service vol- 
untarily by coming to the child support office for an in-person meeting, which then also followed 
principles of behavioral science and procedural justice. 
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The targeted outcome for these interventions was in-person meeting attendance (although phone 
participation was also allowed in VT1). Vermont also hoped to increase the number of orders estab- 
lished by reaching an agreement. Only the GAI test was able to track effects on other outcomes, 
such as order establishment and initial payment rates. 


Highlighted Findings 


= Outreach informed by behavioral science can increase the percentage of parents who 
attend and engage in meetings. 


The GA1 intervention led to an increase in voluntary acceptance of service. The percentage of 
noncustodial parents in the intervention group who came into the child support office to accept 
service voluntarily was 8.2 percentage points (or 54 percent) higher than in the control group. VT1 
also led to a large increase in the proportion of cases in which both parents attended a meeting. 


= Outreach combined with meetings held using principles of behavioral science and pro- 
cedural justice can increase the number of parents who reach stipulation and, poten- 
tially, help them reach agreement sooner. 


The VT1 intervention almost doubled the percentage of orders reached by stipulation. The findings 
suggest that the intervention also sped up the agreement process. Parents in the intervention 
group who reached agreement did so, on average, after just one month. Parents in the control 
group who reached agreement did so after three months, on average. However, the intervention’s 
effect on time to agreement is only suggestive, since it is nonexperimental. 


= Attendance at initial in-person meetings does not necessarily lead to effects on other 
outcomes. 


Although the GA1 intervention brought more noncustodial parents in for meetings to accept service 
voluntarily, it did not have effects on paternity establishment (not shown in the table), order estab- 
lishment, or payment rates. There may not have been effects on these other outcomes because the 
encouragement to accept service voluntarily brought in parents who would have participated in 
establishing orders and made payments anyway, in the absence of the intervention. Alternatively, 
there may not have been effects because there were no intervention components aimed specifical- 
ly at these other outcomes. The VT1 test did not examine effects on other outcomes as part of the 
formal evaluation. 


Engagement After Orders Are Established: Making Payments, 
Particularly Payments on New Orders 


Each of the interventions in this category had the goal of increasing payment rates after orders 
were established, and all but one focused on the first few months after orders were established 
(see Tables 7 and 8). As background, most child support payments are withheld from parents’ pay- 
checks, but it often takes one to three months for income withholding to begin. The main goal of 
these interventions was to increase payments in the first few months after orders were set, before 
income withholding started. Those payments provide critical support to custodial parents and chil- 
dren, and making payments helps noncustodial parents avoid accumulating debt and experiencing 
punitive enforcement actions. 
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Table 7. Summary of BICS Tests Related to Engagement After Orders Are Established 


Making Payments 


Issue 


Initial payment rates 
on newly established 
orders are low. 


Goal Test 


Increase payment C01 
rates in the first few 

months after orders 

are established 


CA2/ 
SAC? 


GA2 


WA2 


TX1 


Intervention 


In-person meeting after order estab- 
lishment to discuss the consequences 
of nonpayment 

A decision tree to help parents deter- 
mine their payment methods 
Payment reminders for three months 
Wallet card with a parent’s payment 
plan 


Explainer sheet listing required next 
steps 

Action-plan implementation prompt 
with payment options outlined 
In-person, phone, text, and email guid- 
ance, informed by behavioral science 
and procedural justice, during the first 
six months of the order, if needed 


Simplified initial payment letter, 
followed by mailed reminders, with in- 
formation on how much parents owed, 
how to make payments, and when 
payments were due 


Telephone outreach to review orders 
with parents, discuss the importance 
of payments, and develop payment 
plans with them 

Letter informed by behavioral science 
laying out next steps and encouraging 
payment 

Text-message payment reminder 


In-person meeting after order estab- 
lishment to discuss the consequences 
of nonpayment and to help parents 

in selecting payment methods and 
making payment plans 
Payment-information materials and a 
decision tree 

A follow-up reminder call one week 
later to allow parents to make pay- 
ments or answer any questions 


Positive 
Effect 


Y 


NA 


(continued) 


WE) °) (Wat (ocolinaliler-xep) 


Positive 
Issue Goal Test Intervention Effect 
Payment rates among Increase the per- CO2 » Payment reminders sent by text mes- N 


parents with estab- centage of parents sage twice per month 
lished orders arelow. — who pay at least 90 

percent of their order 

amounts each month 
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NOTES: The final column indicates whether the evaluation of that intervention detected positive and statistically significant effects 
on at least one outcome of interest. “NA” indicates that the intervention was not evaluated using a randomized controlled trial. 
“This test took place only in Sacramento County. 


Table 8. Impacts on Selected Outcomes Related to Engagement After Orders Are Established 


Making Payments 


Intervention Impact 


CA2/ 
Outcome col SAC? GA2 TX1 co2 
Targeted outcomes 
Paid support in Month 1 (%) 3.7 -3.7 31 4.9* 
Total payments Month 1 ($) 16 -45** 17 
Paid support, Months 1-3 (%) 4.2 -3.2 2.2 2.2 
Total payments Months 1-3 ($) 115** -103** 13 
Paid 90% or more of amount due, Month 1 (%) -2.5* 
Paid 90% or more of amount due, Months 1-3 (%) -1.8 
Net cost ($) 218.88 NA NA 213.82 NA 


SOURCES: MDRC calculations are based on data from the Colorado Division of Child Support Services, the California State 
Department of Child Support Services Data Repository, the Georgia Division of Child Support Services, and the Texas Office of 
the Attorney General. 


NOTES: Estimates in this table are regression-adjusted, controlling for pre-random assignment characteristics. 

Statistical significance levels are indicated as: *** =1 percent; ** = 5 percent; * = 10 percent. 

Cells that are filled gray indicate that impacts were not examined for that outcome. 

Net costs were estimated for each of the interventions in Phase | tests except for Washington, DC and Ohio. Vermont net 
costs were not available due to incomplete data. (Net costs are calculated as the total cost of the intervention per intervention 
group member minus the total cost of business as usual per control group member.) 

“NA” indicates that costs were not estimated. 

"This test took place only in Sacramento County. 
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States use different approaches to inform parents about their child support orders and the avail- 
able methods for making payments. In some states, a proposed order amount is set by the child 
support staff without the noncustodial parent present. If the parent fails to respond to a letter or 
form indicating the proposed order amount, either by accepting the amount or contesting it, the 
order goes into effect by default. In this scenario, noncustodial parents may receive very little infor- 
mation about how to make payments or the consequences of not paying. 


In other states, such as Texas, orders are set during a negotiation meeting in a local office or during 
a hearing at court. However, if the noncustodial parent fails to attend this meeting or hearing, the 
order may still be set without that parent’s participation. When the parent does attend, the meet- 
ing typically focuses on the steps taken to set the order amount, and parents may not receive much 
information about how to make payments. 


The interventions discussed in this section provided parents with information about how to pay 
child support in ways informed by behavioral science, through in-person meetings, phone calls, 
letters, easy-to-read instructions, payment-option tables, wallet cards, and reminders. In the CO1 
and TX1 interventions, for example, staff members provided this information in meetings, while the 
GA2 intervention provided it in a simplified letter sent to parents. 


The Texas intervention targeted employed parents and focused primarily on increasing payments 
in the first few months before income withholding began. Other states targeted a broader group of 
parents. Although they also focused largely on increasing payments in the first few months, states 
hoped these early effects would establish a long-term habit of paying, particularly if income with- 
holding was not put in place. However, none of the interventions measured effects on longer-term 
payment outcomes. 


Highlighted Findings 


= Communications informed by behavioral science, particularly those delivered in per- 
son, can encourage more parents to pay support in the first few months after orders 
are set. 


Both the CO1 and TX1 interventions, which included in-person meetings, led to an increase in initial 
payments. The TX1 intervention increased the number of parents who made payments in the first 
month by 4.9 percentage points, or 9 percent. The CO1 intervention increased payments by $115 
during the first three months. The GA2 intervention, which provided simplified information ina 
letter, also increased the payment rate, but only in the second month. 


= However, outreach informed by behavioral science does not always increase payments. 


The CA2/SAC intervention led to a reduction in payments. The reasons for this effect are unclear, 
but it appears to be largely caused by a reduction in order amounts over time. Finally, the lack of 
effects in the CO2 intervention may suggest that it was not intensive enough, or that effects are 
more difficult to achieve for existing orders than they are for new orders. 
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= Effects may vary depending on the settings in which the meetings are held and the 
types of cases served in these settings. 


The CO1 intervention’s overall positive effect on payments reflects two offsetting effects among 
subgroups. The intervention led to a positive effect among parents whose meetings were held 
during or after administrative order establishment meetings. In contrast, it led to an unexpected, 
negative effect among parents whose orders were established in court (which usually means that 
the parents could not reach agreement in an administrative meeting). It is not clear why the inter- 
vention led to a reduction in payment rates for the latter group, but the difference highlights that 
the same intervention can have different effects depending on the context and the types of cases 
involved. 


= Agencies can integrate order establishment and payment meetings. 


The payment meetings in TX1 and CO1 lasted about 10 to 15 minutes. Some staff members felt that 
it would be feasible and more efficient to integrate these meetings into existing order establish- 
ment meeting. Others, however, saw value in having different staff members handle these different 
functions. 


Engagement After Orders Are Established: Modifying Orders/Registering Online 


Ohio addressed two issues related to engagement after orders had been established (see Tables 
9 and 10). First, parents have the right to request modification of their orders when their financial 
circumstances change or when their orders have not been reviewed in the previous three years. 
However, the modification process in Ohio is long and involves multiple steps parents and staff 
members must take; moreover, many modification reviews are never completed. A parent must 
first apply for a modification and provide paperwork to prove eligibility. If the staff reviews the 
application and finds the parent is eligible, the parent is then required to complete and submit 
additional paperwork. This is a drop-off point for many parents. 


Four of the interventions in Ohio attempted to simplify and improve the modification process 

by making certain parents eligible for reviews by default, simplifying the application and review 
paperwork needed, or providing help to parents along the way. Two of these interventions were 
implemented in Franklin County (OH1/FR and OH2/FR), and two were implemented in Cuyahoga 
County (OH1/CUY and OH2/CUY). The first intervention in each county focused on the eligibility 
stage: OH1/CUY made parents eligible by default and OH1/FR simplified the application paperwork. 
The second set of interventions focused on the review stage, once parents were deemed eligible, 
and included simplified paperwork, the elimination of state paperwork, and the availability of 
dedicated staff members to provide assistance. The targeted outcomes included the percentage of 
cases where modification reviews were scheduled, the percentage where required paperwork was 
submitted, and the percentage where modification reviews were completed. Although the state 
hoped that adjusting orders to reflect parents’ ability to pay would ultimately lead to increased 
payment rates, effects on payment-related outcomes were not examined. 


The second issue with respect to engagement after orders are established involves registering 
online. Ohio offers an online portal for child support, which is designed to let parents manage their 
cases more easily. However, few parents sign up to use it. The final two Ohio interventions (OH3 
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Table 9. Summary of BICS Tests Related to Engagement After Orders Are Established 


Modifying Orders/Registering Online 


Issue 


The order modifica- 
tion process is long 
and complicated, and 
many requests for 
modifications are not 
completed. 


Many parents do not 
use the online tools 
(such as state portals, 
apps, etc.) available 
to them to help man- 
age their cases or find 
information. 


Goal 


Increase the percent- 
age of parents who 
complete the modifi- 
cation process. 


Increase the number 
of parents registered 
on the Ohio online 
portal 


Test 


OH1/ 
CU 


OH1/ 
FRE 


OH2/ 
cuy? 


OH2/ 
FR? 


OH3 


OH4 


Intervention 


Orders made eligible for 
review by default for parents 
who were determined to be 
eligible using administrative 


data 


Simplified application form 
One-page overview and FAQ 


Simplified paperwork 
Phone, text, and letter 


reminders to return the re- 


quired paperwork 


Elimination of additional 


state forms 


Simplified paperwork 


One-page overview and FAQ 
Dedicated, specially trained 
staff to provide outreach and 


assistance 


Elimination of additional 


state forms 


Outreach letter informed by 


behavioral science 


Outreach email informed by 


behavioral science 


Positive 
Effect 


Y 


we 


we 


NOTES: The final column indicates whether the evaluation of that intervention detected positive and statistically signif- 

icant effects on at least one outcome of interest. 
“This test took place only in Cuyahoga County. 
>This test took place only in Franklin County. 
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Table 10. Impacts on Selected Outcomes Related to 
Engagement After Orders Are Established 


Modifying Orders/Registering Online 


Intervention Impact 


OH1/ OH1/ OH2/ OH2/ 
Outcome cuy? FR? cuy? FR? OH3 OH4 
Targeted outcomes (%) 
Modification review 47.9 *** 3.5° 
scheduled 
Modification affidavit 11,3:°"** 31 2.1 16.5*** 
returned 
Modification review 12.4*** 3.27 1.7 15,1*** 
completed 
Registered on portal 1.8*** 3.9*** 
Net cost ($) 0.00 0.00 0.00 0.00 NA NA 


SOURCES: MDRC calculations are based on data from the Cuyahoga County Office of Child Support Services, 
the Franklin County Child Support Enforcement Association, and the Ohio Office of Child Support. 


NOTES: Estimates in this table are regression-adjusted, controlling for pre-random assignment characteristics. 

Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Cells that are filled gray indicate that impacts were not examined for that outcome. 

Net costs were estimated for each of the interventions in Phase | tests except for Washington, DC and 
Ohio. Vermont net costs were not available due to incomplete data. (Net costs are calculated as the total 
cost of the intervention per intervention group member minus the total cost of business as usual per 
control group member.) 

“NA” indicates that costs were not estimated. 

“This test took place only in Cuyahoga County. 

>This test took place only in Franklin County. 


and OH4) attempted to increase the percentage of parents who registered on the portal, through 
outreach by mail and email. 


Highlighted Findings 


= Making parents eligible for a modification review by default led to increases in reviews 
scheduled and completed that were larger than the increases that resulted from only 
simplifying the application paperwork. 


To address the first step in the process, the OH1/CUY intervention made certain parents eligible 
by default and the OH1/FR intervention simplified the application form. The OH1/CUY interven- 
tion nearly doubled the number of cases scheduled for reviews; it also led to large increases in 
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paperwork returned and reviews completed. In contrast, OH1/FR intervention increased reviews 
scheduled by a few percentage points, with only similar-sized increases in paperwork returned and 
reviews completed. 


= Providing specialized staff members to support eligible parents in navigating the pro- 
cess and completing needed paperwork led to effects much larger than only simplify- 
ing the paperwork. 


To address the second stage in the process, the OH2/CUY intervention simplified the needed 
paperwork and sent reminders, once parents were eligible; and the OH2/FR intervention provided 
trained staff members to reach out to parents and help them complete the paperwork. The latter 
intervention led to large increases in the number of forms returned (16.5 percentage points) and 
reviews completed (15.1 percentage points). The former intervention, in contrast, had no statistical- 
ly significant effects on these outcomes. 


THE BICS STRATEGY FOR BUILDING THE CAPABILITIES 
OF CHILD SUPPORT AGENCIES 


The BICS project involved a close collaboration among the participating child support agencies, 
researchers, and OCSE. Each of the grantees conducted behavioral diagnosis and designed, im- 
plemented, and tested their interventions alongside the TAE team, learning the nuts and bolts of 
behavioral science and evaluation first-hand. As part of the BICS award, grantees were responsible 
for identifying a project director and assembling a team to implement and test the interventions. 
The project directors promoted the flow of information among the TAE team, OCSE, and the local 
offices, and served as sources of institutional knowledge. The implementation experience offers 
several lessons for other child support agencies. 


Phases and Approaches 


There were three main phases of the BICS project. In Phase I, the TAE team led the behavioral 
diagnosis and intervention design, and state agencies actively participated in the process. The TAE 
team and each grantee used several tools to identify problems of interest that potentially hindered 
programs from achieving their stated goals. As part of behavioral diagnosis and design in each 
state, the team and grantee worked together to create a process map that laid out the experi- 
ences of the staff members involved in each step of the process, as well as the actions required 

of parents. Using data from the child support agency in that state, the grantee and the TAE team 
conducted a funnel analysis to better understand how cases moved through specific steps in 

the state’s process. The funnel analysis showed the number of cases that made it past each step. 

A drop-off in the percentage of cases that made it past a given step indicated a potential problem 
area. 


The teams then identified behavioral bottlenecks and cognitive biases through interviews with 
parents and staff members. These interviews helped the teams identify potential behavioral expla- 
nations for the observed drop-off points identified in the process map and the funnel analysis, and 
led to ideas for interventions. An example of a behavioral bottlenecks table is shown in Table 11. 
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Table 11. Bottlenecks and Hypothesized Concepts from Behavioral Science 


Case Manager Conference Notification and Scheduling 


CONCEPT FROM BEHAVIORAL SCIENCE 


PROSPECTIVE HASSLE OSTRICH COGNITIVE PROCRASTINATION — NEGATIVE PROCEDURAL 
MEMORY FACTORS EFFECT OVERLOAD AFFECT PLANNING 
FAILURE 


BOTTLENECK 


Notification: parents 

notified about case 

manager conference X X 
one month in advance 


Forms (in notifica- 

tion package): long, 

complicated, invasive; 

require notarization; X X X X 
usually not completed 


Scheduling: meetings 

set during business 

hours; scheduled 

around staff availabil- X X X 
ity; no parent input; 

both parents must 

attend 


NOTE: This table is similar to one developed as part of the multiple iterations of the behavioral diagnosis and design conducted in Vermont 
in Phase I. 


The teams then designed interventions and launched randomized controlled trials to test whether 
the interventions led to desired outcomes. 


In Phase II of BICS, the state child support agencies took the lead in diagnosis and design of a new 
set of interventions, while the TAE team provided support and guidance. Some states implemented 
randomized controlled trials, while others used nonexperimental methods to assess effects. 


In Phase III — the Integration Phase — the child support agencies worked directly with OCSE and 
the Washington Division of Child Support to develop additional interventions and to integrate 
aspects of BICS into their standard practices. During this phase, most state agencies honed their 
diagnosis and design skills further and incorporated their lessons into wider practices. Examples of 
integration activities building on BICS tests are shown in Box 1. 


Box 1. Integrating BICS into Child Support Going Forward 


IN CALIFORNIA’S SACRAMENTO COUNTY, leaders are using behavioral science to change internal 
policies and procedures in ways that take greater account of staff perspectives and experiences. 
For example, the local child support agency has a process to review forms and revise them based 
on principles of behavioral science. An organization-wide training effort took place to encourage 
teams and staff members to develop materials using a perspective informed by behavioral sci- 
ence. This perspective has even reached areas such as facilities management and staff hiring. 


IN CALIFORNIA’S SAN JOAQUIN COUNTY, staff members involved in BICS have become strong 
advocates for the use of concepts from behavioral science when providing services. The agency’s 
goal is to phase in practices informed by behavioral science in all aspects of office operations. All 
employees received training in Behavioral Economics 101 in April 2019. MEF Associates will pro- 
vide additional training to all staff members in May 2019. 


IN GEORGIA, the state is adapting the BICS appointment letter, reminder flyer, and magnets — 
materials from Phase | tests — to be used statewide. Staff members will receive training in behav- 
ioral science to help them understand why the new materials are beneficial and to explore other 
processes where these concepts can be used. A statewide expansion plan for Phase Il materials 
will also be developed. 


IN OHIO, agency changes and decisions are increasingly being viewed through a behavioral- 
science lens. In addition, staff members in counties that were not involved in BICS are now receiv- 
ing training in concepts from behavioral science. 


™ IN CUYAHOGA COUNTY, behavioral diagnosis and design is now a central action step in an 
agency operational plan to help staff identify behavioral bottlenecks such as hassle factors 
and confusing communication. In addition, child support staff members are helping other 
divisions in the state Department of Job and Family Services to implement interventions 
based on behavioral science for participants served by TANF and the Supplemental Nutrition 
Assistance Program. 


= IN FRANKLIN COUNTY, staff members assigned to work on strategic planning and new pro- 
jects are trained in topics related to behavioral economics, as well as the behavioral diag- 
nosis and design process, with the idea that they can apply that training when crafting new 
programs and changes to services. Demand continues to grow both within the agency and 
among community partners for more training and technical assistance in implementing BICS. 


THE VERMONT Office of Child Support is building principles of behavioral science into its every- 
day work in areas ranging from new-employee training to its customer service unit. In addition, 
Vermont is rolling out statewide the resolution meetings tested in Phase | and Phase Il and has 
extended the employer outreach to all employers receiving an income withholding order. The 
office’s strategic plan also signals that behavioral science plays a role in daily operations. 


(continued) 
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Box 1 (continued) 


TEXAS is planning to incorporate principles of behavioral science into program planning and 

is incorporating communications strategies and tools from BICS into ongoing practices. Staff 
members redesigned materials for the most recent tax-time outreach campaign using principles 
of behavioral science “that motivate parents to take a specific action and explain how to do it.” A 
Texas Child Support Division-specific toolkit with worksheets and potential applications across 
agency departments will be developed for use across the state. Local offices have embraced tools 
from the first and second interventions, such as the redesigned appointment-reminder mailers 
and emails, for use even beyond the intervention’s original target population. 


COLORADO is adopting its payment intervention statewide. In addition to the existing materials 
that give noncustodial parents options regarding how to pay their child support, the team has 
developed new information for custodial parents on how to receive their payments. There is also 
a plan to provide small grants and regional coaching in behavioral economics to multiple counties 
that are interested in implementing strategies based on behavioral science. 


THE DISTRICT OF COLUMBIA adopted the new BICS application packet for TANF applicants to re- 
quest child support services. In addition, the child support agency sends text-message reminders 
where possible. All new brochures and outreach materials now use the BICS model. Special care is 
taken to ensure that documents are not too wordy and that processes for making contact with the 
child support office are clearly communicated and hassle-free. 


WASHINGTON plans to develop behavioral-economics training for staff members with the goal 
of incorporating concepts from behavioral science into its work at all levels. Additional training in 
behavioral diagnosis and design will be designed. 


Expanding the BICS Community 


In an effort to broaden the reach of the BICS project beyond the original eight grantees, OCSE 
developed the idea of establishing “BICS peer learning sites.” In the spring of 2018, OCSE invited 
interested state and tribal child support programs to join the BICS community and to develop and 
test interventions based on behavioral science that addressed operational problems of interest 

to them. These peer learning sites consisted of child support agencies from seven states (Indiana, 
Kansas, North Dakota, Michigan, Mississippi, New Jersey, and New York) and three tribes (the Chip- 
pewa Cree Tribe, the Nez Perce Tribe, and the Delaware Tribe of Indians). Each of the peer learning 
sites expressed an interest in behavioral interventions in 2018 and volunteered to participate in 
affinity groups where they applied behavioral diagnosis and design techniques to their problems of 
interest. Peer learning sites attended several workshops with BICS grantees and the BICS Project 
Support Team and then tested their interventions in the final year of the BICS project. Results from 
those tests are being collected by OCSE and disseminated to child support programs interested in 
continuing to learn from the BICS project. This collaboration allowed BICS grantees to act as men- 
tors to peer learning sites, solidifying their own knowledge of behavioral science and the approach 
used in BICS. Table 12 shows the topic areas of interest to peer learning sites and the original BICS 
grantees. 
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Table 12. Peer Learning Sites 


BICS grantees were paired with peer learning sites to provide assistance 
with diagnosing behavioral bottlenecks and designing interventions based 
on problems of interest. 


Affinity Group Topics Peer Learning Sites BICS Grantees 
Payments Indiana Texas 

New Jersey Colorado 

New York 
Noncustodial parent Chippewa Cree California 
engagement Nez Perce Tribe Washington State 
Application and Delaware Tribe Washington, DC 
cooperation Kansas Georgia 


North Dakota 


Complex contacts Michigan Vermont 
Mississippi Ohio 


LESSONS AND CHALLENGES IN IMPLEMENTING 
THE BICS PROJECT 


This project can be a promising model for integrating interventions based on behavioral science 
into child support programs. The BICS project’s unique implementation model combined strong 
child support leadership and dedicated staff with external training and technical assistance sup- 
port (provided by the TAE team early in the project and by OCSE in the later phases). In addition, 
the time-limited nature of the grant funding received from OCSE to implement the project encour- 
aged state child support agencies to focus on implementing this demonstration and make a pri- 
ority of learning and knowledge sharing.” The BICS project was ambitious, launching and testing 
22 interventions in five years as part of the original evaluation scope, with an additional 10 tests 
designed and implemented by peer learning sites. The BICS experience highlights several lessons 
and challenges for implementation. 


= Most interventions took longer to implement than expected. 


There were two main reasons why. First, diagnosis and design took more time than expected. In 
most states, identifying behavioral bottlenecks required tracking a cohort of cases over time to as- 


12 BICS builds on the evidence built from the earlier generation of behavioral science work in social programs in the 
Behavioral Interventions to Achieve Self-Sufficiency (BIAS) project, funded by the Office of Planning, Research, and 
Evaluation in the Administration for Children and Families. For more on the BIAS findings, see Richburg-Hayes, Anz- 
elone, and Dechausay (2017). 
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sess drop-off at each point in the process. While in theory this tracking could be done for old cases 
that had already moved through the process, data for these old cases were sometimes not readily 
available. Some states, given the nature of their desired interventions, did not need to track cases 
over time to identify their main challenges and were able to use data on cases at a point in time 

(a snapshot assessment). But these states were the minority. The second reason implementation 
took longer than expected is that sample enrollment took longer than expected in most 

Phase | tests, almost all of which were randomized controlled trials and required large samples. 
Sometimes sample-enrollment projections were inconsistent with the actual flow of cases. For 
example, some states had fewer cases being opened than were projected based on past data or 
fewer cases where agencies were initiating action than had been estimated. 


= Interventions that involved changing staff practices took more effort to implement 
than those that involved changes to written materials. 


Some of the interventions needed more technical assistance to implement because they required 
state grantees to develop scripts, desk aids, and other tools informed by behavioral science that 
staff members could use to guide their actions. It took time to develop interventions that included 
such components, and it then took more management time and technical assistance to train staff 
members to use the tools. 


= Itis important to create a culture of learning, testing, and sharing knowledge. 


The BICS project encouraged grantees to learn from administrative data, from findings from 
interventions, and from each other, fostering a culture designed to promote creative problem 
solving. The grantees joined learning-community calls to learn about tools based on behavioral 
science as they designed interventions. They attended BICS workshops, presented their ideas, and 
received suggestions about how to improve their proposed tests and learning. In the Integration 
Phase, grantees had an opportunity to practice the skills they learned in BICS by coaching the peer 
learning sites. This practice, in turn, strengthened their abilities to use those skills and to continue 
integrating them into their agency operations. 


= Stable leadership and staff are important. 


Several state agencies experienced staff turnover during the five years BICS operated. When an 
agency’s project director changed frequently, it was difficult for the TAE team to build a relation- 
ship with that agency, and consequently implementation became more difficult. The loss of be- 
havioral science knowledge in the agency also required supplemental staff training in addition to 
project-management transition. Staff turnover meant that new front-line staff members had to be 
trained to conduct random assignment and deliver interventions, which was a large demand on 
their time on top of their regular orientation processes. Planning for staff turnover with succession 
plans and creating solid knowledge-management systems and processes could reduce the disrup- 
tion involved. 


= Change management in large organizations is challenging and takes time. 


Child support agencies have multiple stakeholders — from individual case managers and program 
directors to court staff members and process servers — and each of these stakeholder’s actions 
and approaches can influence case outcomes. Front-line staff members and major actors in state 


child support agencies and in the courts were sometimes skeptical of the new procedures and 
perspectives at first. In addition, while the core BICS staff members generally implemented inter- 
ventions with fidelity, they often had little control over how other staff members behaved. 


= The cost of most interventions was very low relative to business as usual. 


In most states, the added costs of implementing the interventions were quite low, with most being 
under $15 per participant. The costs were higher for interventions that involved in-person meet- 
ings and follow-up calls from staff members. As tested, these increased costs ranged from around 
$214 to $218. A significant portion of these additional costs was tied to new staff members hired 
for grant implementation. The findings from those interventions, and the experience in Georgia, 
suggest that they could be made less expensive if the additional in-person contact were integrated 
into existing meetings. Cost estimates in Texas suggest that implementing the intervention at full 
scale would cost around $12.26 per participant. 


LESSONS FOR CHILD SUPPORT AND BEHAVIORAL SCIENCE 


The BICS interventions focused on multiple stages in the child support process, providing some 
overall lessons and themes. The BICS project and its findings illustrate the promise and limitations 
of interventions informed by behavioral science. 


= Practices informed by behavioral science can be applied at relatively low cost to many 
child support problems, and it is likely that they will produce some positive impacts. 


The interventions targeted a range of outcomes, and most produced positive, albeit small, effects. 
Most were also inexpensive to implement, although they were supported with additional techni- 

cal assistance and monitoring from the TAE team, suggesting that states should consider adopting 
approaches that led to positive outcomes and continue to develop and test new interventions. In 
addition, interventions targeting the modification process are a particularly ripe area for innovation. 


= Creating dedicated, specialized staff units for specific processes has trade-offs. 


There is an ongoing debate within the child support community about staff specialization. On the 
one hand, specialization can help staff members focus on specific tasks, reduce distraction, and 
increase efficiency, particularly for cases that may not require a lot of effort and are suited to amore 
automated approach. However, when staff members specialize based on a particular child support 
process, such as order establishment, cases and parents might get “lost” in the hand-off from one 
staff member to the next, and parents will not have a continuing relationship with, or oversight from, 
a single staff person. Moreover, the cases more likely to get lost may be the more challenging ones, 
which could benefit from an approach in which a single staff person handles the case from begin- 
ning to end. A hybrid approach might be to have specialized staff members work with less compli- 
cated cases and hand over more challenging ones to a dedicated team or staff person. In Ohio, the 
processes for reviewing and modifying cases were improved with greater staff specialization under 
BICS. Similarly, in California, the BICS intervention group had higher service rates, which may have 
been the result of specialized staff members providing order establishment services. Meanwhile in 
Colorado, administrators did not see a benefit to having specialized staff members go over explainer 
materials and payment options with parents during the establishment conference. Agencies can use 
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behavioral diagnosis and design tools to assess the right level of specialization and weigh the pros 
and cons of each approach based on the responses of parents and staff members. 


= Practices informed by behavioral science that are directed at a specific activity will 
probably affect that activity but will not necessarily have effects on other outcomes at 
later points in a process. 


Interventions focused earlier in the child support process (such as service or order setting) are like- 
ly to affect outcomes at those points in the process (responses to service or agreements reached) 
but may not affect payment rates. However, the evidence on this point is limited and comes from 
only a few interventions. Although most states hypothesized that positive effects on early engage- 
ment would lead to better outcomes at each step in the process, most of the tests were not de- 
signed to see if there were effects on payments. 


= Practices informed by behavioral science that are incorporated into existing child sup- 
port processes rather than being applied as new and separate processes can reduce 
the burdens on staff members and parents. 


Several child support staff members indicated that if principles of behavioral science are to be suc- 
cessfully integrated into standard practices, they should be incorporated into management struc- 
tures that staff members have already accepted. This integration would also signal that principles 
of behavioral science were now a lasting part of an agency’s culture. One of the grantees created a 
part-time change-management position to incorporate insights from behavioral science into reg- 
ular practice, which helped streamline processes but also signaled to staff members that leaders 
thought those insights were important. 


= In-person meetings early in the process can help set the stage for a productive, coop- 
erative relationship between parents and a child support agency. 


The findings, along with views of participating staff, suggest that initial, in-person meetings pres- 
ent an opportunity for staff members to connect with parents in a more helpful manner, allowing 
staff members to tailor services to individual cases. For example, instead of sending overly com- 
plicated and lengthy information in the mail, staff members can use the meeting that parents are 
already required to attend to provide this information. Although in-person contacts are labor-in- 
tensive, costs could be minimized by using techniques drawn from behavioral science to make 
existing meetings more effective. For example, if administrators want to add more in-person time, 
they should consider adding that new time to a meeting that is already part of the process. This 
approach would remove the hassle factors for parents that could result from a new meeting and 
would allow staff members to make more of existing meetings once parents are in the room. 


= Practices informed by behavioral science cannot address nonbehavioral barriers. 


Parents do not engage with the child support process for a variety of reasons, some of which can 
be addressed by interventions informed by behavioral science. But many low-income noncusto- 
dial parents may not be able to pay support because of unemployment, incarceration, or very low 
earnings. These are not behavioral issues, and therefore cannot be solved through interventions 


that draw only on behavioral science.” That said, efforts to address these and other nonbehavio- 
ral barriers can be aided by insights from behavioral science. For instance, a child support agency 
could use insights learned from behavioral science to increase participation in a noncustodial 
parent employment program. 


CONCLUSION 


The BICS project demonstrates that behavioral science has significant potential to improve child 
support services for parents and staff members. The behavioral diagnosis and design process 
provides a systematic, evidence-informed approach to identify and tackle some of the problems 
parents and staff members in child support agencies experience. The interventions tested in this 
project cost relatively little and produced some gains at all points in the child support process. 
Moreover, the BICS project demonstrated that agencies could implement new practices, test 
them, and refine them as needed, in the spirit of continuous improvement. Each of the participat- 
ing states has incorporated concepts from behavioral science into its program going forward. In 
addition, the states and OCSE are already demonstrating the ability to mentor additional states 
and localities as they attempt to improve their services and meet their goal of securing support for 
children. 


Consistent with earlier research, the BICS findings demonstrated that tools such as personaliza- 
tion, simplification, and reminders can have significant effects on program participants’ behavior, 
and that these tools typically affect near-term outcomes. As the field of behavioral science ma- 
tures, additional research can assess when behavioral interventions are appropriate, compared 
with more intensive, structural changes. In addition, further research can identify and refine a new 
generation of tools informed by behavioral science that might lead to positive long-term outcomes. 


13 For additional research, see Dubey (1995). 
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Appendix A 


Summaries of the BICS Tests 


Randomized controlled trial. Sample of 3,906 individ- 
uals, 1,975 in the intervention group and 1,931 in the 
control group across Sacramento and San Joaquin 
Counties. Study cases were followed for 12 months us- 
ing California’s child support administrative records. 


Personalization 
Simplification 
Implementation prompts 


Loss aversion 


DESIGN 


The ultimate goal of the intervention was to increase 
engagement in the order establishment process as 
evidenced by an increase in the percentage of orders 
established by stipulation or hearing and a decrease 
in default orders. The immediate goal was to increase 
the percentage of parents who submitted an Answer 
form (used by parents to respond to service). The in- 
tervention targeted newly opened cases. Noncustodi- 
al parents in the intervention group received outreach 
materials informed by behavioral science, scripted 
phone calls, and case management from specialized 
caseworkers. This outreach occurred before orders 
were established. 


INTERVENTION COMPONENTS 


Specialized BICS case managers trained in princi- 
ples of behavioral science and procedural justice to 
handle order establishment 


Service package adjusted using principles of be- 
havioral science, adding an explainer sheet as a 
cover sheet (in English or Spanish as appropriate) 
and moving the Answer form: 


The Answer form moved from the middle of the 
service package to the top. 


The explainer cover sheet provided clear instruc- 
tions for parents opening the service package 
and suggested next steps. 


Scripted calls from specialized caseworkers after 
parents were served to discuss next steps 


Scripted calls after Answer forms were submitted 
to acknowledge parents’ actions and encourage 
additional engagement 


IMPACT FINDINGS 


The intervention had an unexpected but positive 
impact on service rates (in part due to smaller case- 
loads and more specialized duties among staff mem- 
bers serving in the intervention group), increasing 
the percentage of parents who received service by 
5.0 percentage points, from 69.6 percent to 74.6 per- 
cent.’ Other postservice impacts may result from the 
fact that more intervention group parents were suc- 
cessfully served (rather than resulting from the inter- 


1 Foramore detailed discussion of this unexpected finding, see Box 2 in Gaffney, Fishman, and Smith (2019). 


California Phase | (CA1) Intervention (continued) 


vention itself). This increase in service rates probably 
occurred in part because specialized staff members 
provided order establishment services for intervention 
group cases. 


The intervention found effects on outcomes including 
the percentage of cases with Answer forms submit- 
ted (a 3.1 percentage point increase) and the percent- 
age in which orders were established. There was also 
a Statistically significant $93 — or 26.5 percent — in- 
crease in total payments on study orders in the first 12 
months after random assignment. Overall establish- 
ment rates increased by 17.1 percentage points among 
Spanish-speaking parents and by 4.1 percentage points 
among English-speaking parents. The proportion of or- 
ders established in hearings or by agreement (rather 
than by default) increased by 21.5 percentage points 
among Spanish-speaking parents; there was no statis- 
tically significant effect among English-speaking par- 
ents. 


The intervention had no impact on the percentage of 
cases in which any payment on current support was 
made in the first year. The proportion of current sup- 
port paid was also similar between the intervention 
and control groups over the first 3, 6, and 12 months 
following study enrollment, although these three com- 
parisons are nonexperimental. 


IMPLEMENTATION FINDINGS 


The intervention components were largely implement- 
ed as planned, with three main exceptions: (1) BICS 
case managers in San Joaquin County made additional 
efforts to locate parents when they were not success- 
fully served while case managers in Sacramento Coun- 
ty did not (which may explain why service rates were 
higher in San Joaquin than in Sacramento); (2) BICS 
case managers in both counties did not always call 
parents after they returned their Answer forms if they 
felt the second call was unnecessary; and (3) BICS case 
managers in both counties incorporated intervention 
materials into other aspects of their work. 


Staff members and managers appreciated the BICS 
approach, saying that it encouraged a more open flow 
of information between staff members and parents, 
increasing understanding for parents and leading to a 
culture shift among the BICS case managers. 


ESTIMATED NET COSTS: 
= Sacramento County: $7.65 per intervention group 
member 


= San Joaquin County: $14.46 per intervention group 
member 


= Average net cost per intervention group member 
across both counties: $11.06 


Randomized controlled trial. Sample of 1,638 indi- 
viduals, 790 in the intervention group and 848 in the 
control group, enrolled over a 13-month period. Study 
cases were followed for six months using California’s 
child support administrative records. 


Implementation prompts 
Loss aversion 
Personalization 


Simplification 


Design: The goal of this intervention was to increase 
the percentage of payments made in the first few 
months after order establishment. The intervention 
targeted newly established orders. Noncustodial par- 
ents in the intervention group received in-person out- 
reach and print materials informed by behavioral sci- 
ence. This outreach occurred immediately after order 
establishment. 


INTERVENTION COMPONENTS 


Specialized BICS case managers, trained in princi- 
ples of behavioral science and procedural justice, 
who provided services after order establishment. 


Case managers made scripted calls, sent text 
messages, and sent emails after order establish- 
ment to develop payment plans with parents or 
encourage them to adhere to those plans. 


For orders established in court, interviews with 
parents after hearings conducted by BICS case 
managers 


Meetings were guided by talking points and a 
checklist. Case managers encouraged each par- 
ent to make and commit to a specific payment 
plan. Staff members filled out payment plans 
with parents. 


An explainer cover sheet encouraging early payment 


The cover sheet provided clear instructions for 
parents with newly established orders, regard- 
less of order type. 


Parents also received a wallet card payment plan 
card to fill out independently. 


IMPACT FINDINGS 


The intervention had no impact on the percentage of 
cases where payments were made in the first three 
months following order establishment or on the pro- 
portion of current support paid in the first six months. 
It reduced rather than increased total payments in the 
first six months by $228. On average, control group 
participants paid $1,652 while intervention group par- 
ticipants paid $1,424. 


It is possible that higher total payments among the 
control group were the result of higher order amounts. 
After one month, control group participants owed on 
average $440 the following month, while interven- 
tion group members owed on average $407. After six 
months, control group participants owed on average 
$393 the following month, while intervention group 
members owed on average $337. It is unclear why the 


California (Sacramento) Phase II (CA2/SAC) Intervention 


(continued) 


difference between control and intervention group 
participants’ order amounts grew during the follow-up 
period. 


IMPLEMENTATION FINDINGS 


BICS case managers were more likely to identify pay- 
ments as the goal of the interviews held after hearings 
than control group case managers. However, when 
researchers observed the interviews, they saw little 
distinction in the way the two groups of case manag- 
ers approached the meeting, nor in the talking points 
they used. The primary distinction was that the BICS 
case managers emphasized payment planning and 
used the wallet card. 


BICS case managers felt the talking points were most 
useful in the interview held after hearings and less 
useful for phone outreach. However, they also felt that 
even with the talking points and improved materials, 
it was difficult to make concrete payment plans with 
noncustodial parents after hearings because parents 
often claimed they would pay “as soon as possible” 
rather than committing to a specific date. 


BICS staff members and parents were especially en- 
thusiastic about the new wallet card implementation 


prompts. Parents observed and interviewed after 
their post-hearing meetings said that they found the 
wallet cards helpful and would keep them somewhere 
prominent, as intended. 


The Phase II test represented an opportunity for the 
county to embed a rigorous experiment within agen- 
cy operations. Agency leaders took several steps to 
maintain experimental conditions in the office, includ- 
ing making sure that the caseworkers assigned to BICS 
cases and the control group were evenly matched 
based on their past performance, and physically iso- 
lating the BICS caseworkers within the office to pre- 
vent spillover. However, because staff members were 
not aware of the matching method based on past per- 
formance, it is possible that control group casework- 
ers may have expended extra effort on collections to 
prove that they were as “strong” as the intervention 
group that had been selected to implement the inter- 
vention. According to agency leaders, this byproduct 
of implementing an experiment may have contributed 
to the null and negative payment outcomes. 


ESTIMATED NET COSTS 
Not calculated for this intervention. 


Randomized controlled trial. Sample of 2,476 individ- 
uals, 1,222 in the intervention group and 1,254 in the 
control group, enrolled over a 16-month period. Study 
cases were followed for 3 to 18 months using Califor- 
nia’s child support administrative records. 


Personalization 
Simplification 
Implementation prompts 


Loss aversion 


DESIGN 


The goal of the intervention was to increase parental 
involvement in order establishment. The intervention 
targeted newly opened cases. Noncustodial parents 
in the intervention group received the same print out- 
reach materials informed by behavioral science as in 
Phase |, without the additional staff involvement." 


INTERVENTION COMPONENTS 


A cover sheet for the service package (in English or 
Spanish as appropriate), with the Answer form — 
used by parents to respond to the service package 
— moved from the middle of the service package to 
the top 


1 See the earlier description of the CA Phase | intervention (CA1). 


The explainer cover sheet provided clear instruc- 
tions for parents opening the service package 
and suggested next steps. 


Print materials were the same as Phase | materi- 
als except for the contact information. The Phase | 
materials provided a dedicated phone line to the 
BICS case workers, while these Phase II materials 
directed parents to a call center. 


IMPACT FINDINGS 


The intervention did not affect the percentage of An- 
swer forms submitted. While the intervention did in- 
crease the percentage of orders established in the 
intervention group by 3.4 percentage points, from 
55.0 percent to 58.4 percent, this increase can prob- 
ably be attributed to the intervention’s unexpected, 
positive impact on orders established by default. The 
intervention increased the percentage of orders es- 
tablished by default by 4.0 percentage points, from 
36.6 percent to 40.6 percent, but it had no effect on the 
percentage of orders established by hearing or stipu- 
lation, the targeted order types. The intervention did 
increase total child support payments in the first three 
months by $17, on average, from $20 to $37. It did not 
increase payments over the first six months. 


As was the case in the Phase | test, the intervention 
had an unexpected, positive impact on service rates, 
increasing service rates by 4.3 percentage points, from 
60.5 percent to 64.8 percent. However, because there 
were no specialized staff members providing order 


California (San Joaquin) Phase II (CA2/SJ) Intervention 
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establishment services for intervention group cases, 
it is more plausible that this increase was due to extra 
efforts (conscious or subconscious) made by the pro- 
cess servers when serving packages with the BICS cov- 
er sheet, though this explanation is difficult to prove. 


Because there is an impact on service but no impact 
on Answer forms submitted as there was in Phase l, itis 
possible that the Phase | impact on Answer forms sub- 
mitted can be attributed to the efforts of the specialized 
BICS caseworkers (absent in the Phase II research de- 
sign), and was not an incidental outcome of increased 
service rates alone. According to agency leaders, it is 
difficult to get parents to submit Answer forms. The 
Phase II study results suggest that modifications to the 
service package alone may not be enough to produce 
an impact in the absence of dedicated caseworkers to 
follow up with parents after the package is delivered. 


IMPLEMENTATION FINDINGS 


The intervention components were implemented as 
planned, with one exception related to data collection: 
Because staff members did not routinely collect infor- 


mation about incoming communication from control 
group parents, the impact study was not able to in- 
clude that outcome measure. 


Since staff members did not have to be actively in- 
volved in the Phase II test, most were unaffected by 
its implementation. Leaders felt the second test was 
successful because the new changes were more in line 
with how people think and, as a result, have begun ed- 
ucating staff members about the principles of behavio- 
ral science underlying the Phase | and II tests. 


Leaders also appreciated the opportunity presented 
by the Phase II test to examine other agency process- 
es (for example, the agency’s call center phone tree) 
through the lens of behavioral science, and they re- 
ported that they feel there are clear benefits to imple- 
menting practices informed by behavioral science in 
those other processes in the future. 


ESTIMATED NET COSTS 
Not calculated for this intervention. 


Randomized controlled trial. Sample of 697 individ- 
uals, 342 in the intervention group and 355 in the 
control group. Study cases were followed for three 
months using child support administrative records; 
implementation, time study, and financial data; and 
observations collected during site visits. 


Implementation prompts 
Loss aversion 
Personalization 
Reminders 


Simplification 


DESIGN 


The goal of the intervention was to increase payment 
amounts and the percentage of parents who made 
payments after order establishment. The intervention 
focused on cases with newly established orders. Non- 
custodial parents in the intervention group received 
outreach materials informed by behavioral science. 
When noncustodial parents in the intervention group 
came tothe office forthe Administrative Process Action 
(APA) meeting, they received an enhanced in-person 
meeting; if they went to a court hearing, staff mem- 
bers attempted to administer the intervention over 
the phone. During the office meeting, the intervention 
group received personalized and simplified materials, 


and if they went to the court hearing, these materials 
were mailed to them. They also received follow-up 
monitoring and payment reminders for the three 
months following order establishment. 


INTERVENTION COMPONENTS 
Payment meeting with caseworker 


The caseworker and the parent reviewed an info- 
graphic that illustrated the consequences of not 
making full payments. The parent was guided 
through an online decision tree that recommend- 
ed a payment method based on the parent’s 
payment habits, and filled out a wallet card that 
contained payment information and an imple- 
mentation plan. 


Personalized reminders sent by text, email, or 
phone (depending on a parent’s preference) 


Reminders included a notification that payment 
was due, an indication of the parent’s desired 
payment method, a payment-identification num- 
ber, and the phone number of a caseworker who 
could answer questions. 


Follow-up monitoring 


Caseworkers called parents if they missed pay- 
ments. 


IMPACT FINDINGS 


There was a $115 (or 19.9 percent) increase in the 
amount paid by parents within the first three months, 
from $579 for the control group to $694 for the inter- 
vention group. This difference is statistically signifi- 


Colorado Phase | (CO1) Intervention (continued) 


cant. There was no effect on the percentage of parents 
who made payments within three months of order 
establishment, which was close to 67 percent in both 
groups. 


The effect of the intervention varied across subgroups. 
Among cases established through an administrative 
process, there was a $197 impact on the amount par- 
ents paid and a 10.7 percentage point impact on the 
proportion of parents who made payments. Among 
cases established in court there was no impact on 
the amount paid and a -16.7 percentage point impact 
on the proportion of parents who made payments. 
Among parents who had more than one case, there 
was a positive impact of $405 on the amount paid and 
no impact on the percentage who made payments. 
Among parents who had only one case, there was no 
impact on the amount paid or on the percentage who 
made payments. 


There was no impact on the child support debt balance 
six months after order establishment. A nonexperimen- 
tal analysis suggests that the intervention reduced the 
time it took parents to make their first payments and 
increased the use of the autopay payment method. 


IMPLEMENTATION FINDINGS 


In some counties, caseworkers incorporated the pay- 
ment meetings into the APA conferences rather than 


holding them afterward. In some counties, casework- 
ers who conducted the APA conferences handed par- 
ents off to another caseworker, who then conduct- 
ed the payment meeting. Caseworkers responded 
positively to the intervention, reporting that parents 
seemed to find the payment meetings, infographic, 
online decision tool, and wallet card useful. However, 
parents who were dissatisfied with their child support 
orders tended to respond negatively to the payment 
meeting and intervention materials. 


It was not feasible to implement the intervention at 
court because very few parents attended their court 
hearings. The intervention would need to be rede- 
signed to be implemented at court in the future. In-per- 
son payment meetings took 15 minutes; meetings over 
the phone took longer because caseworkers took time 
to describe intervention materials. 


Caseworkers conducted variable outreach after the 
intervention. Some caseworkers sent text messages 
and emails and called parents to remind them to make 
payments each month, and some caseworkers called 
parents only when they missed payments. 


ESTIMATED COSTS 


$218.88 per intervention group participant, based on a 
time study conducted in three of the four counties. 


Randomized controlled trial. Sample of 5,210 individ- 
uals, 3,477 in the intervention group and 1,733 in the 
control group. Study cases were followed for three 
months using state child support administrative 
records. 


Implementation prompts 
Personalization 


Reminders 


DESIGN 


The goal of the intervention was to increase the per- 
centage of parents who made full payments each 
month. Noncustodial parents in the intervention group 
received two-way text-message payment reminders 
at the beginning and end of each month if they had not 
made full payments.' If they had made full payments, 
then they received “thank-you” text messages at the 
end of the month. Parents in the control group did 
not receive additional outreach other than potentially 
receiving the state’s one-way payment-reminder text 
message, which became part of the status quo during 
the study period.? Some of the counties had addition- 
al outreach strategies for parents in the intervention 


group, such as calling parents or sending emails, but 
those outreach methods varied among caseworkers 
and counties. 


INTERVENTION COMPONENTS 
Payment-reminder text message 


On the fifth of every month, parents were sent 
two-way payment-reminder text messages if 
they had not made full payments in the previous 
month. The messages asked if parents planned 
to make the payments and asked them how they 
planned to make payments, prompting them to 
choose from a list of payment options. Payment 
information was provided based on the selected 
method. A second reminder message was sent on 
the twentieth of the month to parents who had 
still not made full payments. 


“Thank-you” text message 


Parents were sent “thank-you” text messages on 
the twentieth of the month if they had made full 
payments. 


IMPACT FINDINGS 


The intervention led to a 2.5 percentage point reduc- 
tion in the number of cases where parents made full 
payments in the first month. This reduction is mar- 
ginally statistically significant at the 10 percent level 
and was unexpected. However, there were no impacts 


1 Ifa parent paid 90 percent or more of the amount owed, that was considered a “full payment.” 
2 One-way texting allowed the agency to send parents reminders, but parents could not send a message back. Two-way texting allowed the 
agency and parents to text back and forth and was believed to be more helpful to parents because they could ask questions by text. 
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on the percentages of cases where parents made full 
payments in Months 2 or 3. There was no impact on 
the percentage of cases where parents ever made full 
payments in Months 1 through 3, nor on the percentage 
where parents always made full payments in Months 1 
through 3. 


In addition, although the intervention was initially 
supposed to have another arm to be tested separately 
(consisting of outreach to employers by child support 
staff members), there were challenges with putting this 
additional component and test into operation, and the 
two intervention arms were combined into one for the 
purposes of analysis. 


IMPLEMENTATION FINDINGS 


Due to delays in implementing the intervention and 
complications with data acquisition, the team did not 
collect implementation research data, and there are no 
implementation research findings to report. For exam- 
ple, the research team does not know the types of ser- 
vice referrals provided to parents by text nor how often 
those referrals were provided, and there are no data on 
the kinds of interactions that occurred between par- 
ents and technicians. 


ESTIMATED NET COSTS 
Not calculated for this intervention. 


Randomized controlled trial. Sample of 635 individu- 
als, 318 in the intervention group and 317 in the control 
group. Study cases were followed for three months us- 
ing agency child support records. 


Deadlines 
Identity priming 
Personalization 
Reminders 


Simplification 


DESIGN 


The goal of the intervention was to engage parents 
receiving Medicaid services. The intervention target- 
ed unmarried custodial parents on Medicaid. Parents 
in the intervention group received outreach material 
and reminders. If a parent responded requesting child 
support services, that parent was sent a redesigned 
and streamlined child support services application. 
Cases were followed for three months after the initial 
outreach began. 


INTERVENTION COMPONENTS 


Outreach letter informed by behavioral science 
that asked parents to choose between (1) estab- 
lishing paternity or receiving child support services 
and (2) closing their cases 


Robocall reminding custodial parents to respond 
to the letter 


Reminder postcard 


A postcard informed by behavioral science was 
sent to nonresponders that gave parents the 
same two choices as the initial letter. 


Redesigned child support services application 


Asimpler, shorter child support application using 
principles of behavioral science was sent to par- 
ents who requested it. 


IMPACT FINDINGS 


The proportion of parents who made contact with 
the child support agency increased by 4 percentage 
points, from 3 percent in the control group to 7 per- 
cent in the intervention group. There was a 2 per- 
centage point increase in the number of parents who 
submitted child support services applications, from 
0 percent in the control group to 2 percent in the in- 
tervention group. There were no significant impacts 
on the percentages of cases where child support pe- 
titions were generated, where legal paternity was es- 
tablished, or where parents requested that their cases 
be closed. The child support agency was able to close 
the cases of nonresponders after making multiple at- 
tempts to reach them, which decreased the agency’s 
outstanding caseload. 


IMPLEMENTATION FINDINGS 


The response rate to the robocalls was very low be- 
cause many phone numbers on file for parents were 
missing, disconnected, or incorrect. Setting up the 
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robocalls required the agency to work with outside ESTIMATED NET COSTS 
departments, which was time-consuming. Around 3 Not calculated for this intervention. 
percent of mailings were returned to sender, indicating 

that the address was incorrect. 


A nonexperimental analysis of outcome measures be- 
fore and after the intervention — a pre/post test. The 
318 individuals in the sample were originally in the 
control group in the DC1 test. Study cases were fol- 
lowed for three months using Washington, DC child 
support administrative records. To assess the effects 
of this test, the means of outcome measures for the 
parents who were in the control group in the first BICS 
DC intervention (DC1) were compared with the means 
of the same parents in the second time period (DC2) 
Therefore, this analysis used a pre/post method of 
comparing the same population with itself at two dif- 
ferent time periods: after the intervention of DC1 and 
after the intervention of DC2. This was a nonexperi- 
mental test and therefore it cannot confirm with cer- 
tainty that the BICS intervention was the cause of any 
apparent effects. 


Deadlines 
Identity priming 
Personalization 
Reminders 


Simplification 


DESIGN 


The goal of this intervention was to engage parents in 
older child support cases where parents were receiv- 
ing Medicaid services. This test targeted control group 
cases from the DC1 test and was implemented a few 
months after DC1. The intervention was an enhanced 
version of the one in DC1 that included texting out- 
reach and the option to reply to the letter and postcard 
by text message. DC2 used the same intervention ma- 
terials as DC1, except the outreach letter and postcard 
were adapted to invite the custodial parent to request 
child support services by text message. Additionally, 
an introductory text message was sent to those custo- 
dial parents whose phone numbers the child support 
agency had, and a reminder text message was sent if 
they did not respond. The child support agency had 
phone numbers for parents in only about 20 percent 
of the cases. 


INTERVENTION COMPONENTS:? 
Outreach letter 
A letter informed by behavioral science was sent 
asking parents to choose between (1) establish- 


ing paternity or receiving child support services 
and (2) closing their cases. 


Introductory text message introducing the child 
support agency to parents 


Reminder postcard 


1. Originally, the intervention design also included robocall reminders for parents to respond to the outreach letter, but due to technical 


issues these robocalls never occurred. 


District of Columbia Phase | (DC2) Intervention Test 2 


(continued) 


© A postcard informed by behavioral science was 
sent to nonresponders that gave parents the same 
two choices as the initial letter. 


= Reminder text asking nonresponders to reply 


Findings 


IMPACT FINDINGS 


The proportion of parents who made contact with 
the child support agency increased by 11 percentage 
points, from 3 percent in time period 1 to 14 percent in 
time period 2. There was a1 percentage point increase 
in the number of parents who submitted child support 
services applications, from 0 percent in period 1 to 1 
percent in period 2. There were no changes in the per- 
centages of cases that had child support obligations 


established, where legal paternity was established, or 
where a parent requested that the case be closed. 


IMPLEMENTATION FINDINGS 


Staff members reported that texting was simple to im- 
plement, and that it was easy to respond to text mes- 
sages using a desktop interface. However, because 
these cases were relatively old many of the phone 
numbers used for the texting were not active and the 
text messages did not go through. Around 1 percent of 
mailings were returned to sender, indicating that the 
address was incorrect. Parents were not interviewed 
about this intervention. 


ESTIMATED NET COSTS 
Not calculated for this intervention. 


This intervention was designed as a randomized con- 
trolled trial. However, due to changes in the manage- 
ment information system used by the TANF office, 
cases were not transferred to the child support office, 
delaying the intervention for multiple years. Once the 
technical problem with case transfers was addressed, 
it turned out that cases appropriate for the evaluation 
sample were building up too slowly, and the test was 
cancelled. 


Identity priming 
Loss aversion 
Personalization 
Reminders 


Simplification 


DESIGN 


The goals of this intervention were to persuade custo- 
dial parents to open child support cases so that they 
would not lose TANF benefits, and to increase the 
number of parents who attended an intake meeting. 


INTERVENTION COMPONENTS 
Outreach letter 


The letter welcomed a custodial parent to the 
child support agency and provided information 


about the upcoming appointment that was set 
up for the parent to open a child support case af- 
ter he or she was referred by the TANF agency. 


Encouragement text message 


The message described the benefits of child 
support. 


Reminder text messages 


Two days before the appointment, a text mes- 
sage was sent asking the custodial parent to con- 
firm that he or she would attend the appointment 
and inviting the parent to reschedule if not. 


The morning of the appointment, a text message 
was sent reminding the parent that the appoint- 
ment was that day and that he or she should 
bring documents relating to marriage, divorce, 
child support, and paternity. 


Missed-appointment message or missing-docu- 
ment message 


On the day after the scheduled appointment, a 
text message was sent to the custodial parent 
asking him or her to reschedule (if the parent did 
not attend) or to send a clear photo of required 
documents (if the parent did not bring any of 
them to the appointment). 


Missed-appointment postcard 


On the day after a missed appointment, a postcard 
was sent encouraging the parent to reschedule. 


Follow-up text message 


About 50 days after a case was transferred to the 
child support office, if a custodial parent did not 
attend an appointment that parent received a 


District of Columbia Phase II (DC3) Intervention 
(continued) 


text message saying that his or her TANF benefits 
were now Sanctioned and he or she would start to 
lose money. Additionally, the message said that he 
or she could stop losing money by responding and 
rescheduling the appointment. 


Findings 


IMPACT FINDINGS 


No analysis was conducted because of the technical is- 
sues described above. 


IMPLEMENTATION FINDINGS 


There were unanticipated challenges with interdepart- 
mental coordination that had major effects on the in- 
tervention and evaluation. The interface with the TANF 


office’s database did not work and ultimately led to this 
intervention not being evaluated. 


In human services agencies, there are often major in- 
ternal barriers in the process to approve work with out- 
side vendors such as text-messaging providers. These 
delays should be included in the planning for rollout 
of an intervention that involves text messaging. The 
intervention components are still being implemented 
by the grantee, and while they are not being evaluat- 
ed, staff members report that they are having positive 
effects. 


ESTIMATED NET COSTS 
Not calculated for this intervention. 


Randomized controlled trial. Sample of 2,931 indi- 
viduals, 1,463 in the intervention group and 1,468 in 
the control group. Study cases were followed for six 
months using state child support records. 


Deadlines 
Implementation prompts 
Loss aversion 
Personalization 


Simplification 


DESIGN 


The goal of the intervention was to engage noncus- 
todial parents early in the establishment process and 
increase the percentage who accepted service in the 
office voluntarily. The intervention targeted people 
who had recently been named as parents in a child 
support case. Parents in the intervention group re- 
ceived outreach materials informed by behavioral sci- 
ence. If they came to the office, they also had an en- 
hanced in-person meeting informed by principles of 
procedural justice. 


INTERVENTION COMPONENTS 
Redesigned notice in an eye-catching envelope 


The hand-addressed brown envelope stood out in 
the mail. The notice was simpler than the previous 
letter and had features to support planning and 
highlight the financial consequences of inaction. 


Calendar magnet with personalized appointment 
information 


The magnet was included with the notice as an 
added aid in planning. 


Reminder notice mailed days before an appointment 


This notice included a short list of documents to 
bring, a map, and a phone number to call to re- 
schedule if needed. 


Enhanced in-person meeting 


Specially trained staff members conducted meet- 
ings using a script and checklist to make sure they 
consistently incorporated principles of proce- 
dural justice and completed important meeting 
components. 


Staff members were trained to help parents 
request a reduction in the child support order 
amount (called a low-income deviation adjust- 
ment) if there was evidence that the parent 
earned a low income or was unemployed. 


IMPACT FINDINGS 


There was an 8.2 percentage point increase in the 
proportion of parents who were served in the office 
voluntarily, from 15.1 percent of the control group to 
23.3 percent of the intervention group. This difference 
is statistically significant and represents a 54 percent 
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increase. However, there was no effect on overall cas- 
es served; close to 60 percent of both groups received 
service. 


The effect of the intervention varied among the three 
offices that participated. The largest office, in an ur- 
ban setting, had an 8.6 percentage point impact on the 
number of parents who were served in the office, and 
a 6.3 percentage point impact on overall cases served. 
The smallest office in the study, in a rural setting, had 
a 14.2 percentage point impact on service in the office, 
but a negative impact on overall cases served. It is un- 
clear why the intervention would have decreased over- 
all service rates there. There was no effect on either 
outcome in the third office, which was in a midsized 
city. 


Caseworkers holding meetings with parents in the in- 
tervention group did not seem to make more requests 
for low income deviation adjustments compared with 
the control group. Average order amounts were similar 
between the two groups after the intervention period. 


IMPLEMENTATION FINDINGS 


Random assignment processes and study protocols 
were mostly followed as planned. 


Most administrators and staff members felt the new 
meeting procedure was an improvement. They found 
the approach “nonintimidating” and “eye-opening,” 
and said they believed it made parents feel more com- 
fortable. Several parents told researchers they felt that 
they were treated fairly and respectfully. 


ESTIMATED NET COSTS 
= The total net cost of the intervention, including staff 
time and materials, is $9.02. 


= The total cost of materials for each intervention 
group member is $3.75, while the net cost of materi- 
als for this intervention is $2.58. 


=" Most of the net cost ($6.44) reflects the added staff 
costs associated with the intervention. 


Randomized controlled trial. Sample of 1,788 individu- 
als, 900 in the intervention group and 888 in the con- 
trol group. Study cases were followed for three months 
using state child support administrative records. 


Deadlines 
Implementation prompts 
Loss aversion 

Reminders 


Simplification 


DESIGN 


The goal of the intervention was to increase the rate of 
first payments made within 30 days after order estab- 
lishment. The intervention targeted cases with newly 
established orders for support. Noncustodial parents 
in the intervention group received outreach materials 
informed by behavioral science. 


INTERVENTION COMPONENTS 
Redesigned notice in an eye-catching envelope 


The larger envelope was designed to stand out in 
the mail. The notice was simpler than the previous 
letter and included the parent’s name, the child’s 
name, the amount due, and clear instructions on 
how to make a payment. A self-addressed enve- 


lope with a postage stamp was included, along 
with a colorful payment-options flyer. 


Reminder notice mailed three to five days before 
the due date 


The notice was mailed in a brown envelope and 
included the parent’s name, the child’s name, the 
amount owed, and instructions on how to make 
a payment. The notice addressed the common 
misconception that manual payments are not 
necessary until income withholding takes place. 
The letter was simplified and written at a seventh- 
grade level. 


Late-notice reminder mailed 10 days after the due 
date if full payment was not received 


The simplified notice had two main messages: 
(1) your payment is now late, which could lead to 
hassles later; and (2) make contact with the office 
for help if you cannot make a payment right now. 


IMPACT FINDINGS 


The intervention did not affect on-time payments 
(defined as payments within 30 dates of the effective 
due date). Analyzing the effect on payments within 33 
days of the effective payment date (to account for due 
dates that fell on a weekend or holiday) did not change 
results. 


The intervention did lead to a 5.1 percentage point in- 
crease in payments in the second month after random 
assignment. This impact is statistically significant at 
the 5 percent level. There were no effects on payments 
in other months. 


Georgia Phase II (GA2) Intervention (continued) 


The intervention led to a small increase in the aver- 
age number of months in which a payment was made 
during the first three months, from 1.2 for the control 
group to 1.3 for the intervention group. This impact is 
statistically significant at the 10 percent level. 


IMPLEMENTATION FINDINGS 


Very few parents used the self-addressed envelopes. 
The office received only a handful of self-addressed en- 
velopes with payments. 


ESTIMATED NET COSTS 
Not calculated for this intervention. 


Randomized controlled trial. Sample of 3,267 individ- 
uals, 1,633 in the intervention group and 1,634 in the 
control group. Study cases were followed for at least 
three months using administrative records from Cuy- 
ahoga County Office of Child Support Services. 


Simplification 


DESIGN 


The goal of the intervention was to increase the pro- 
portion of parents who reached important milestones 
in the modification review process: scheduling mod- 
ification reviews, returning affidavits to the agency, 
and completing the modification review process. Cuy- 
ahoga County eliminated the first step in the order 
modification process — the eligibility-screening ap- 
plication — for parents whose eligibility for modifica- 
tions could be determined using existing administra- 
tive data. Essentially, Cuyahoga started modification 
reviews by default for parents meeting certain criteria: 
(1) they were incarcerated or (2) they were inquiring 
about modifying orders that had not received modifi- 
cation reviews in the previous 36 months. 


INTERVENTION COMPONENTS 
For some parents, removing the eligibility screen- 
ing step that requires parents to fill out paperwork 
to demonstrate eligibility for modification 


IMPACT FINDINGS 


There was a 47.9 percentage point increase in the pro- 
portion of cases that were scheduled for review, from 
49.0 percent in the control group to 97.0 percent in the 
intervention group. This increase led to effects in the 
second stage of the process: The proportion of cas- 
es that saw modification affidavits (another round of 
paperwork parents had to complete) returned to the 
agency increased by 11.3 percentage points, from 29.4 
percent to 40.7 percent. And the proportion of cases 
that saw reviews completed increased by 12.4 percent- 
age points, from 31.6 percent to 44.0 percent. All differ- 
ences are statistically significant at the 1 percent level. 


Descriptive statistics show that the Test 1 intervention 
decreased the time it took to get a review scheduled 
by 38.8 percent, or by 43.4 days. A nonexperimental 
analysis of the Test 1 cases that received modifications 
found that parents in the intervention group had to 
wait an average of 68 days after random assignment 
to receive new order amounts, while parents in the 
control group had to wait 112 days on average. 


IMPLEMENTATION FINDINGS 


Though implementation was initially challenging 
due to staff turnover and the county’s need to make 
changes to its information technology to deliver the 
intervention, after the first few weeks the test was de- 
livered as designed. 


ESTIMATED NET COSTS 


The county reported no increases in costs for the in- 
tervention group over the control group. 


Randomized controlled trial. Sample of 2,565 individ- 
uals, 1,270 in the intervention group and 1,295 in the 
control group. Study cases were followed for at least 
three months using administrative records from the 
Franklin County Child Support Enforcement Association. 


Simplification 


DESIGN 


The goal of the intervention was to increase the pro- 
portion of parents who reached important milestones 
in the modification review process: scheduling modifi- 
cation reviews, returning affidavits to the agency, and 
completing the modification review process. Parents 
in the intervention group received a simplified modifi- 
cation application package with a redesigned, double- 
sided form and a one-page fact sheet that included 
language to encourage parents to complete the form. 


INTERVENTION COMPONENTS 
A graphically redesigned, clarified, and simplified 
application form that used clearer language in- 
formed by behavioral science 


The packageincluded aone-pagedocumentwitha 
simplified, four-step overview of the modification- 
review process on one side and Frequently Asked 
Questions on the other. 


It also included an easy-to-read table of eligibility 
requirements, with an explanation of each and an 
example of the documents required. 


IMPACT FINDINGS 


In the intervention group, 42.1 percent of modifica- 
tion requests had reviews scheduled, compared with 
38.6 percent in the control group. The difference of 
3.5 percentage points is statistically significant at the 
10 percent level, and an increase of 9.1 percent over 
the control group. These differences carried through 
the review process: The intervention increased the 
proportion of modification requests that saw mod- 
ification affidavits (another round of paperwork par- 
ents had to complete) returned to the agency by 3.1 
percentage points (or a 16.4 percent increase) and in- 
creased the proportion of modification requests that 
saw reviews completed by 3.2 percentage points (a 
14.8 percent increase). Descriptive results show that 
the test very modestly increased the amount of time it 
took for orders to complete the modification process. 


IMPLEMENTATION FINDINGS 
The intervention was largely implemented as designed. 


ESTIMATED NET COSTS 


Franklin County reported reduced costs for the inter- 
vention group compared with the control group. 


Randomized controlled trial. Sample of 3,895 individ- 
uals, 1,927 in the intervention group and 1,968 in the 
control group. Study cases were followed for at least 
three months using administrative records from the 
Cuyahoga County Office of Child Support Services. 


Simplification 
Loss aversion 
Personalization 
Reminders 


Implementation prompts 


DESIGN 


The goal of the intervention was to increase the pro- 
portion of parents who reached important milestones 
in the modification review process: returning affida- 
vits to the agency and completing the modification re- 
view process. The intervention provided parents with 
greatly simplified modification paperwork along with 
reminders to complete it. Additionally, the state sup- 
pressed its own version of the affidavit for parents in 
the intervention group, so only one version was sent 
to those parents (parents in the control group contin- 
ued to receive two versions — county and state — ei- 
ther of which could be completed). 


INTERVENTION COMPONENTS 
Revised affidavit packet 


A one-page modification form was sent, along 
with a fact sheet that highlighted the option to 
fast-track the modification review if both parents 
agreed. 


Follow-up calls 


Approximately five days after the revised affidavit 
was mailed, agency staff members attempted to 
reach intervention group parents by phone using 
a script. 


Additional follow-up 


Depending on the contact information available, 
intervention group parents were also sent letters 
or text messages or given automated telephone 
reminders approximately 10 days before the dead- 
line for returning the form. 


IMPACT FINDINGS 


The number of affidavits returned and reviews com- 
pleted increased by about 2 percentage points, but 
these increases are not statistically significant, mean- 
ing that the differences cannot be attributed to the in- 
tervention with confidence. 


IMPLEMENTATION FINDINGS 


A nonexperimental comparison of modification re- 
quests in the intervention group showed that 60.2 
percent of modification requests completed the mod- 


Ohio Phase | Cuyahoga Test 2 (OH2/CUY) (continued) 


ification process when either parent was reached by 
phone, compared with 54.3 percent of modification 
requests when neither parent was reached by phone. 


ESTIMATED NET COSTS 


The county reported no increase in costs for the inter- 
vention group over the control group. The county re- 
ported that savings from Test 1 (OH1/CUY) more than 
offset any costs associated with texting and reminder 
calls made by staff members in Test 2. 


Randomized controlled trial. Sample of 1,249 individ- 
uals, 614 in the intervention group and 635 in the con- 
trol group. Study cases were followed for at least three 
months using administrative records from the Frank- 
lin County Child Support Enforcement Association. 


Simplification 


Reminders 


DESIGN 


The goal of the intervention was to increase the pro- 
portion of parents who reached important milestones 
in the modification review process: returning an affi- 
davit to the agency and completing the modification 
review process. Franklin County combined dedicated 
staffing with simplified materials. Parents in the Test 
2 intervention group were sent a simplified four-page 
version of the order modification paperwork, known 
as an affidavit, and the same overview and Frequent- 
ly Asked Questions document used in Test 1 (OH1/FR). 
The state suppressed the state version of the affidavit 
for parents in the intervention group, so only one ver- 
sion was sent to parents (parents in the control group 
continued to receive both the state and the county 
versions, either of which could be completed). Addi- 
tionally, the county created a specialized modification 
unit to assist parents in the intervention group with 
their applications. 


INTERVENTION COMPONENTS 
A graphically redesigned, clarified, and simplified 
application form that used clearer language in- 
formed by behavioral science 


The package included a one-page document with 
a simplified, four-step overview of the modifica- 
tion review process on one side and Frequently 
Asked Questions on the other. 


It also included an easy-to-read table of eligibility 
requirements, with an explanation of each and an 
example of the documents required. 


A specialized modification unit 


These specialized staff members reached out to 
parents who had modification review dates set 
and offered them assistance to complete the 
forms (in person, by phone, or by email, depend- 
ing on their preference) 


IMPACT FINDINGS 


In the intervention group, 58.3 percent of the modifi- 
cation requests with reviews scheduled saw affidavit 
packets returned, compared with 41.9 percent in the 
control group, an increase of 16.5 percentage points (or 
39.4 percent). This difference is statistically significant 
at the 1 percent level. Furthermore, nearly 63.5 per- 
cent of the intervention group’s modification requests 
completed the modification process, compared with 
only 48.4 percent in the control group, an increase of 
15.1 percentage points (or 31.2 percent). This difference 
is statistically significant at the 1 percent level. A non- 
experimental analysis of the length of time it took to 
complete the process showed it took cases in the in- 


Ohio Phase | Franklin County Test 2 (OH2/FR) (continued) 


tervention group slightly longer to complete the modi- 
fication process than cases in the control group. 


IMPLEMENTATION FINDINGS 


Test 2 was largely implemented as designed. In inter- 
views, members of the dedicated modification unit 
reported that they felt well prepared to provide the in- 
tervention, found it professionally rewarding, and be- 
lieved that it was more effective in helping parents. 


ESTIMATED NET COSTS 


The county reported reduced costs for the intervention 
group compared with the control group. Because the 
intervention group Test 2 mailings were shorter, the 
county used less paper and ink. The county reallocated 
staff members, caseloads, and functions among its en- 
forcement units to create the two-person modification 
unit. This reallocation had to be cost-neutral for it to be 
approved by the county administration. 


Randomized controlled trial. Sample of 4,193 individu- 
als, 1,869 in the intervention group and 2,324 in the con- 
trol group. Study cases were followed for seven weeks 
using child support agency administrative records. 


Simplification 
Personalization 
Implementation prompts 


Social influence 


DESIGN 


The goal of the intervention was to increase the per- 
centage of parents registered on the state’s online 
child support portal through mailings. The interven- 
tion targeted parents who were not registered on the 
portal and for whom the agency did not have email 
addresses. Parents in the intervention group received 
a letter informed by behavioral science. 


INTERVENTION COMPONENTS 
Outreach letter 


A personalized letter demonstrating the benefits 
of registering on the portal was sent to parents. 
The letter used social influence and also provid- 
ed clear instructions about how to register on the 
portal. 


The Cuyahoga County version of the letter used 
bright colors such as orange and yellow while 
Franklin County’s letter used blue and gray. 


IMPACT FINDINGS 


There was a 1.8 percentage point increase in the pro- 
portion of parents registered on the portal, from 1.7 
percent of the control group to 3.4 percent of the in- 
tervention group. This difference is statistically signif- 
icant at the 1 percent level and represents a 100 per- 
cent increase. 


IMPLEMENTATION FINDINGS 


The counties found this intervention easy to imple- 
ment. There were some client contacts resulting from 
the calls. The intervention had a greater impact in 
Cuyahoga County, which also had a larger percent- 
age of control group parents register. This difference 
in impacts may have occurred because Cuyahoga 
used bright colors in its letter while Franklin County 
used darker colors, or it may have occurred because 
the parents in Cuyahoga might be more receptive to 
any type of outreach. A subsequent test revealed that 
emails were more effective than letters at getting par- 
ents to register on the portal. 


ESTIMATED NET COSTS 
Not calculated for this intervention. 


Randomized controlled trial with nine research groups. 
Sample of 12,465 individuals, 11,099 in the eight inter- 
vention groups and 1,366 in the control group. Study 
cases were followed for four to five weeks using child 
support agency administrative records. This evalu- 
ation used a factorial design where eight versions of 
the email template were used to test the effects of 
different messages. The control group did not receive 
any outreach. A factorial design is an experimental de- 
sign that allows a researcher to observe the effect of a 
specific factor on an outcome by randomly assigning 
individuals to different combinations of intervention 
components. This design also can test for interaction 
effects: whether the effect of one component varies 
depending on the presence or absence of the other 
components. 


Simplification 
Personalization 
Implementation prompts 
Social influence 


Reminders 


DESIGN 


The goal of the intervention was to increase the per- 
centage of parents registered on the state’s online 
child support portal through email. The intervention 


targeted parents who were not registered on the on- 
line child support program portal but whose email 
addresses were known to the program. Parents in the 
intervention groups received an email informed by 
behavioral science. A second reminder email that con- 
tained the same information was sent one week later. 


INTERVENTION COMPONENTS 
Outreach email 


A personalized email demonstrating the benefits 
of registering on the portal was sent to parents. 
The email used social influence and also provid- 
ed clear instructions about how to register on the 
portal. 


Eight draft email templates were used, testing 
the effects of elements directed at three distinct 
concepts that have been identified as influencing 
behavior in some past research: attention, moti- 
vation, and implementation. The first set of ele- 
ments drew parents’ attention to the portal and 
made them more aware of it and its benefits. The 
second used social influence to motivate them to 
register, and the third provided easy implementa- 
tion steps to register on the portal. 


IMPACT FINDINGS 


There was a 3.9 percentage point increase in the pro- 
portion of parents who registered on the portal, from 
4.0 percent of the control group to 7.9 percent of the 
intervention group. This difference is statistically sig- 
nificant at the 1 percent level and represents a 98 per- 
cent increase. The factorial analysis did not conclude 
that any factor or combination of factors had a greater 


Ohio Phase II Emailing Test (OH4) (continued) 


impact on registration rates than any other; the impact 
seems to be related to receiving an email rather than 
not receiving one. 


IMPLEMENTATION FINDINGS 


The counties found that this intervention was not as 
easy to implement as the mailing intervention (see 
OH3) because mass emailing technology was new 
for them. The counties found the email intervention 
prompted more client contacts than the mailing inter- 
vention, probably because it was easier for someone to 
reply to an email than it was to call or email in response 
to a letter. This extra contact was an additional ben- 
efit, since it allowed staff members to speak with cli- 


ents about both the portal registration and other case 
issues. The counties faced different constraints in de- 
termining how to deliver the email intervention. Cuy- 
ahoga County (which had a larger sample size) opted 
to send emails only to custodial parents, while Franklin 
County chose to email both noncustodial parents and 
custodial parents. 


ESTIMATED NET COSTS 
Not calculated for this intervention. 


The original design was to have a randomized con- 
trolled trial. However, due to process issues that were 
not uncovered during the diagnosis and design phase, 
sample build-up was both slower than expected and 
unevenly distributed between the intervention and 
control groups. As a result, the grantee did not move 
forward with the test. 


Simplification 
Personalization 
Implementation prompts 


Social influence 


DESIGN 


The goal of the intervention was to increase the per- 
centage of parents registered on the state’s online 
child support portal through in-person meetings. This 
intervention focused on parents who were sched- 
uled to attend in-person hearings with agency staff 
members and who were not already registered on the 
portal. Parents in the intervention group received a 
scripted message and personalized instructional ma- 
terial for registering on the portal, both informed by 
behavioral science. They were also offered help if they 
wanted to register for the portal while they were on- 
site, and received follow-up text messages. 


INTERVENTION COMPONENTS 
In-person meeting 


Staff members had a script informed by behavio- 
ral science that they used to promote the portal. 


Parents received a handout promoting the portal. 


Parents were offered an opportunity to register 
on the spot. 


Parents received a series of follow-up text mes- 
sages reminding them to register on the portal. 


IMPACT FINDINGS 


Data from this intervention were not analyzed be- 
cause of the recruitment issues mentioned above. 


IMPLEMENTATION FINDINGS 


The two counties implementing the test faced differ- 
ent constraints. Franklin County determined early 
on that it did not have the operational capabilities to 
run the test as designed. Cuyahoga County opted to 
launch the test but determined later that the test on 
the whole was not viable. However, it did adopt some 
materials from the Phase II tests into its standard op- 
erations going forward. 


ESTIMATED NET COSTS 
Not calculated for this intervention. 


Randomized controlled trial. Sample of 1,167 individu- 
als, 582 in the intervention group and 585 in the con- 
trol group. Study cases were followed for three months 
using agency child support administrative records. 


Simplification 
Implementation prompts 
Public/private commitment 
Reminders 


Loss aversion 


DESIGN 


The goal of the intervention was to increase payments 
in the initial months after order establishment. The in- 
tervention focused on newly established cases. Non- 
custodial parents in the intervention group attended 
a meeting focused on the payment process, and re- 
ceived a follow-up call one week later. 


INTERVENTION COMPONENTS 
Payment meeting 
Noncustodial parents had one-on-one meetings 


with case managers immediately after order es- 
tablishment. 


Materials informed by behavioral science 


During the payment meeting, case managers pro- 
vided written materials to help parents under- 
stand their payment options and decide how to 
make payments: a welcome letter, a decision tree, 
a debt-accrual graphic, a wallet card, a payment- 
option table, and one page of details on the cho- 
sen payment option. 


Follow-up call from the case manager 


One week after the payment meeting, case man- 
agers called parents to remind them to make pay- 
ments and to answer any questions they had. 


IMPACT FINDINGS 


There was a 4.9 percentage point increase in the pro- 
portion of parents who made payments in the first 
month after order establishment (an increase from 
56.5 percent to 61.4 percent). This difference is statisti- 
cally significant at the 10 percent level and represents 
a 9 percent increase. There was no effect on the pro- 
portion of parents who made payments in the second 
and third months, which was close to 70 percent in 
both groups. 


The intervention had different effects among noncus- 
todial parents who had one child support order (the 
one established that day) than it did among those 
who had other orders, and thus had previous experi- 
ence with the child support system. The intervention 
produced an impact of 6.5 percentage points in first- 
month payments among parents with just one order 


Texas Phase | (TX1) Intervention (continued) 


and no impact among parents with multiple orders. 
Among the four regions that participated in the study, 
the largest region had a 9.1 percentage point impact on 
payment rates in the first month. In the other regions, 
the impacts on payment rates are not statistically sig- 
nificant. There were no statistically significant impacts 
on the total average amount parents paid in the first 
three months. 


IMPLEMENTATION FINDINGS 


The intervention was adapted slightly to fit each re- 
gion’s population and processes. Most intervention 
group members participated in the payment meetings, 
which lasted 10 minutes on average. It was difficult for 
staff members to reach parents for follow-up calls. 


ESTIMATED NET COSTS 


= $214 per intervention group participant, almost all 
for additional staff time related to program imple- 
mentation. 


=" This amount is much more than an equivalent inter- 
vention would cost if it were integrated into stand- 
ard practice in the state. It would cost an estimat- 
ed $12.26 more per parent than business as usual 
to implement the intervention as part of standard 
practice. 


A nonexperimental analysis of outcome measures be- 
fore and after the intervention — a pre/post test. Sam- 
ple of 4,425 child support meetings scheduled with 
parents over three months in seven offices. Outcomes 
for each office were measured using agency child sup- 
port administrative records. 


Simplification 
Personalization 
Loss aversion 


Social influence 


DESIGN 


The goal of this intervention was to increase the num- 
ber of orders agreed to in the office. Parents with ap- 
pointments to establish their orders in the child sup- 
port office received outreach materials informed by 
behavioral science. For six of the seven offices, these 
materials consisted of a flyer in the mail; for the sev- 
enth office, the information was sent by email when 
email addresses were available. 


INTERVENTION COMPONENTS 
Flyer/email informed by behavioral science 
The flyer/email featured bright colors and includ- 


ed pictures of smiling kids to attract attention 
and put parents in a positive state of mind. 


It highlighted the consequences of not attending 
the meeting. 


It included quotes from parents who had attend- 
ed in-office meetings about the benefits of meet- 
ing in the office rather than in court. 


It provided information about what to bring and a 
direct telephone number for parents to call if they 
had questions. 


Special child support case managers to answer 
calls 


These case managers fielded calls from parents 
who received the flyer or email. 


IMPACT FINDINGS 


The pre/post test results are inconclusive. Three of- 
fices experienced increases in the percentage of ap- 
pointments that led to agreements in the office, while 
two offices experienced reductions and two offices 
experienced little difference. 


IMPLEMENTATION FINDINGS 


The child support case managers fielded 395 calls 
from parents. The most common reason parents 
called was to reschedule their appointments. The next 
most common reasons were to update contact infor- 
mation, ask questions following the meeting (for those 
who attended), and ask questions about the process. 


Among a sample of 34 parents who came to the meet- 
ing and said they had received the flyer or email, most 
said they would have come to the meeting even if they 
had not received the material. At the same time, they 


Texas Phase II (TX2) Intervention (continued) 


all reacted positively to the flyer or email. Several said 
that it helped reduce the stress they had about the 
meeting and others said that it provided useful infor- 
mation, such as what materials they should bring with 
them. 


Parents had positive reactions to the new design. A 
subset of 55 parents were surveyed and asked for ex- 
amples of things that stood out as positive or negative 
in the materials; most of the respondents pointed out 
more than one positive example, and the few negative 


reactions were about the child support process, not 
about the materials. 


Fewer customers than expected called the direct 
phone number provided to them. 


ESTIMATED NET COSTS 
Not calculated for this intervention. 


Randomized controlled trial. Sample of 947 cases, 464 
in the intervention group and 483 in the control group. 
Study cases were followed for at least eight months 
using state child support records. 


Simplification 

Reminders 

Priming exercise 
Implementation prompts 
Personalization 


Loss aversion 


DESIGN 


The goal of this intervention was to increase the partic- 
ipation rate in meetings outside of court and increase 
the rate of agreements reached between parents in 
those meetings. The intervention targeted cases in 
the order establishment stage and modification stage 
of the child support process. Parents in the interven- 
tion group received outreach materials informed by 
behavioral science, followed by a Resolution Meeting 
that used principles of procedural justice. 


INTERVENTION COMPONENTS 


Redesigned initial packet in an eye-catching 
envelope 


A large envelope was used to stand out in the 
mail. A cover sheet highlighted the most impor- 
tant items to bring to the meeting and suggest- 
ed parents place the required documents in the 
envelope to keep them organized. The welcome 
letter in the packet was simpler than the one the 
agency had been using before. It included a map 
that showed the location of the office, directions, 
the contact information of the case manager 
meeting with the parent, and a calendar image 
with the appointment date and time circled. 


Reminder calls five days and one day before the 
Resolution Meeting 


Case managers used scripts aimed to make the 
calls friendly, helpful, and respectful while they 
reinforced the importance of attending the meet- 
ing. Parents had the option of rescheduling the 
meeting to fit their schedules. 


Resolution Meeting, an administrative alternative 
to an existing court-based conference 


The case managers who led these meetings were 
trained in principles of procedural justice to in- 
crease parents’ understanding of the process, 
provide parents with a voice, remain neutral, and 
be helpful and respectful. This new meeting was 
longer than the existing conference and was of- 
fered at more flexible hours. 


Priming exercise right before the Resolution Meeting 


Parents were offered an optional exercise that 
asked a series of questions designed to get them 
to think about their identity as parents. 


Vermont Phase | (VT1) Intervention (continued) 


Findings 


IMPACT FINDINGS 


Participation in the targeted meeting (the Resolution 
Meeting for the intervention group and the existing 
conference for the control group) by both parents in- 
creased by 8.4 percentage points, from 26.3 percent of 
the control group to 34.7 percent of the intervention 
group (a 31.9 percent increase). This impact is statisti- 
cally significant at the 1 percent level. The proportion 
of cases where both parents reached agreement in- 
creased by 11.3 percentage points, from 12.4 percent 
of the control group to 23.7 percent of the intervention 
group (a 91.1 percent increase). This impact is statisti- 
cally significant at the 1 percent level. 


The length of time from the initiating action (case 
opening or an application for modification) to agree- 
ment decreased by 70 days among cases where agree- 
ment was reached at a targeted meeting. (This finding 
is nonexperimental.) 


IMPLEMENTATION FINDINGS 


The intervention was mostly delivered as designed. 
There were slight revisions made at the beginning to 
make the intervention compatible with case managers’ 
workloads. Roughly half of the parents chose not to 
complete the priming activity meant to precede their 
Resolution Meetings after learning that it was option- 
al. Several parents told researchers they liked the new 
materials and found them helpful and inviting. 


ESTIMATED NET COSTS 


A preliminary analysis indicated that the intervention 
services may have been less costly than business as 
usual due to a reduction in service of process for cas- 
es that were resolved or dismissed outside of court, as 
well as a reduction in cases being filed with the court. 
These anticipated savings are from OCS’s perspective 
and are largely the result of early parent engagement 
and stipulations occurring earlier in the process. Pre- 
cise estimates are not available due to limited data. 


Nonexperimental comparison group. Sample of 578 
cases, with 293 cases receiving transportation cards 
in Phase II and 285 cases from Phase | that did not re- 
ceive the transportation cards. This study compared 
participation rates by both parents in the intervention 
group in Phase | with participation rates by both par- 
ents in the intervention group in Phase Il. The study 
used data from child support administrative records, 
and data were analyzed by Veritas HHS. 


The intervention used all the same strategies as in 
Phase I, plus a monetary incentive in the form of 
$25 VISA cards for transportation. Parents did not 
have to provide any proof of expenses incurred. 


DESIGN 


The goal of the intervention was to increase participa- 
tion rates in Resolution Meetings by offering transpor- 
tation cards to parents who come to the Resolution 
Meetings. This intervention is an iteration of the one 
used in Phase |. 


INTERVENTION COMPONENTS 
Transportation card 


In addition to the intervention components from 
Phase I, transportation cards for gas and public 
transit were provided as incentives to parents. 
These $25 VISA cards were mentioned in the ini- 
tial letter and reminder calls made to parents. 


IMPACT FINDINGS (FROM AN ANALYSIS 
CONDUCTED FOR VERMONT BY VERITAS HHS) 


There was a 5.5 percentage point increase in partici- 
pation in two out of the three regions where the inter- 
vention was tried. (This increase is an average across 
the two regions.) There was a 6.1 percentage point in- 
crease in the agreements reached between parents or 
cases being dismissed because an action was not ap- 
propriate. (This increase is also an average across the 
two regions.) 


ESTIMATED NET COSTS 
Not calculated for this intervention. 


Randomized controlled trial. Sample of 1,855 individu- 
als, 884 in the intervention group and 971 in the con- 
trol group. Study cases were followed for six months 
using agency child support administrative records. 


Personalization 
Deadlines 

Simplification 
Implementation prompts 


Salience 


DESIGN 


The goal of this intervention was to increase parent 
engagement early in the order establishment pro- 
cess. The intervention targeted newly opened cases 
to engage parents in the order establishment process. 
Before orders were established, noncustodial parents 
in the intervention group received phone calls from 
specialized case managers and outreach materials in- 
formed by behavioral science. 


INTERVENTION COMPONENTS 
Calls from specialized case managers 


Case managers trained in principles of behavioral 
science and procedural justice made pre-service 
phone calls guided by talking points and check- 
lists. 


Personalized follow-up letter if phone contact did 
not occur 


If case managers did not reach parents by phone, 
case managers sent personalized follow-up let- 
ters encouraging them to call the agency. 


Service package cover sheet 


Case managers added a cover sheet to the front 
of each service package sent to parents that pro- 
vided clear instructions, a simple summary of the 
proposed order amount, and an implementation 
prompt about what the recipient should do next. 


IMPACT FINDINGS 


A higher proportion of intervention group members 
received consent orders than control group members 
(5 percent compared with 2 percent).' This difference 
is statistically significant. However, there was no effect 
on other targeted outcomes, such as the percentage 
of parents who requested hearings after their orders 
were established and the percentage of orders estab- 


1 Consent orders occur when one or both parents object to a proposed order amount but the parents come to an agreement without requir- 
ing a finding by an administrative law judge. Consent orders typically occur during or immediately before a hearing. 


Washington Phase | (WA1) Intervention (continued) 


lished through agreed settlements.” The intervention 
also did not find impacts on child support payments. 
There were no statistically significant impacts on av- 
erage total payments or on the percentage of parents 
who made some payments within the first six months 
after random assignment. There was also no effect on 
timely service (the percentage of cases served within 
90 days). A lack of effect on timely service suggests 
that, contrary to the concerns of many case managers 
in the agency, more active outreach approaches in ad- 
vance of service did not delay the order establishment 
timetable. 


IMPLEMENTATION FINDINGS 


The intervention components were largely implement- 
ed as planned, though rates of contact with parents 
were lower than expected, with approximately half of 
the intervention group having phone contact with a 
BICS case manager. 


BICS case managers reported that they appreciated the 
more active, customer-oriented approach. They liked 
the opportunity to engage with customers before their 
orders were established, feeling that it allowed them to 
establish a more collegial, less adversarial dynamic. 


Intervention group parents interviewed by the BICS 
team said they appreciated the caseworkers’ ability to 
explain information in a clear and helpful way, but the 
parents did not generally report improved perceptions 
of the child support agency overall. 


ESTIMATED NET COSTS 
= $11.09 per intervention group participant. 
= Most of the net cost ($10.78) reflects the added la- 


bor costs associated with the intervention; printing 
costs were minimal. 


2 Similar to consent orders, agreed settlements occur when one or both parents object to a proposed order amount but parents come to an 
agreement without requiring a finding by an administrative law judge. Agreed settlements typically happen before a hearing takes place. 


Implementation study. Sample of 299 individuals en- 
rolled over seven months at one field office. 


Personalization 
Simplification 
Implementation prompts 


Reminders 


DESIGN 


The goal of the intervention was to increase payments 
in the early months after order establishment. The in- 
tervention targeted parents with newly established 
orders. A specialized group of caseworkers called par- 
ents and sent print materials to parents to encourage 
them to make voluntary and consistent payments. In 
addition, parents who could be reached by phone and 
who agreed to text reminders were sent text messages 
prompting them to follow through with the payment 
plan they discussed on the call. 


INTERVENTION COMPONENTS 


Telephone outreach immediately following order 
establishment 


Specialized BICS case managers made phone 
calls guided by talking points and a checklist, 
encouraging parents to make specific payment 
plans and commit to them. 


Order establishment letter 


Whether or not they made phone contact with 
parents, case managers sent parents a letter en- 
couraging them to review their orders, make pay- 
ments, and call or email the agency with ques- 
tions. 


Reminder text message one day before payment 
was due 


IMPLEMENTATION FINDINGS 


BICS case managers made contact with about 43 per- 
cent of parents by phone or text. During implemen- 
tation, BICS case managers began notifying parents 
by text message that they would be calling, in order 
to increase phone contact rates. After case managers 
adopted this “text-first” approach, the contact rate in- 
creased from 40 percent to 51 percent. Case managers 
were able to send initial text communication requests 
to 63 percent of parents enrolled after the “text-first” 
switch (57 out of 90 parents had viable phone num- 
bers), and 37 percent of those who received text-mes- 
sage requests (21 out of 57) then called their casework- 
ers directly, before the caseworkers called them. 


Overall, 9 percent of parents agreed to and were sent 
subsequent payment-reminder text messages (20 per- 
cent of those reached by phone). 


According to the grantee project director, staff mem- 
bers appreciated the opportunity to give parents no- 
tice by text message that they would be calling, and 
felt that they were more likely to make contact with 
parents whom they texted first. Parents generally 


Washington Phase II (WA2) Intervention (continued) 


responded positively to the calls, but staff members 
found that many parents wished to discuss the terms 
of their orders rather than making payment plans, 
which made it difficult to use the talking points. Staff 
members also found that parents who had questions 
about the terms of the orders or who did not agree 
with them were also less likely to agree to payment- 
reminder text messages. 


ESTIMATED NET COSTS 
Not calculated for this intervention. 
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